Re: [Python-Dev] PEP 418: Add monotonic clock

[Victor Stinner]

>>  * time.time() = system clock
>>  * time.monotonic() = monotonic clock
>>  * time.hires() = monotonic clock or fallback to system clock
>>
>> time.hires() definition is exactly what I was trying to implement with
>> "time.steady(strict=True)" / "time.try_monotonic()".
>
> Please don't call the fallback version "hires" as it suggests it may
> be higher resolution than time.time() and that's completely the wrong
> idea.

Why would it be a wrong idea? On Windows, time.monotonic() frequency
is at least 1 MHz (can be GHz if it uses your CPU TSC) whereas
time.time() is only updated each millisecond at the best case (each 15
ms by default if I remember correctly).

On UNIX, CLOCK_MONOTONIC has the same theorical resolution than
CLOCK_REALTIME (1 nanosecond thanks to the timespec structure) and I
expect the same accuracy.

On Mac, I don't know if mach_absolute_time() is more or as accurate
than time.time().

time.hires() uses time.monotonic() if available, so if
time.monotonic() has an higher resolution than time.time(),
time.hires() can also be called a high-resolution clock. In practice,
time.monotonic() is available on all modern platforms.

> If we're simplifying the idea to only promising a monotonic
> clock (i.e. will never go backwards within a given process, but may
> produce the same value for an indefinite period, and may jump forwards
> by arbitrarily large amounts),

I don't know any monotonic clock jumping "forwards by arbitrarily
large amounts". Linux can change CLOCK_MONOTONIC speed, but NTP
doesn't "jump".

> then we're back to being able to enforce monotonicity even
> if the underlying clock jumps backwards due to system clock
> adjustments.

Do you know a monotonic clock that goes backward? If yes, Python might
workaround the clock bug directly in time.monotonic(). But I would
prefer to *not* workaround OS bugs.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Victor Stinner]

>> Scott> monotonic_clock = always goes forward but can be adjusted
>> Scott> steady_clock = always goes forward and cannot be adjusted
>>
>> I don't know if the monotonic clock should be called time.monotonic() or
>> time.steady(). The clock speed can be adjusted by NTP, at least on Linux
>> < 2.6.28. (...)
>
> You are right that CLOCK_MONOTONIC can be adjusted, so the Boost
> implementation is wrong. I'm not sure that CLOCK_MONOTONIC_RAW is right
> either due to suspend -- there doesn't appear to be a POSIX or Linux
> clock that is defined that meets the "steady" definition.

The term "adjusted" should be clarified. A clock can be adjusted by
setting its counter (e.g. setting the system date and time) or by
changing temporary its frequency (to go faster or slower). Linux only
adjusts CLOCK_MONOTONIC frequency but the clock is monotonic because
it always goes forward. The monotonic property can be described as:

t1=time.monotonic()
t2=time.monotonic()
assert t2 >= t1

> In that case, I don't think time.try_monotonic() is really needed
> because we can emulate "time.monotonic()" in software if the platform is
> deficient.

time.hires() is needed when the OS doesn't provide any monotonic clock
and because time.monotonic() must not use the system clock (which can
jump backward).

As I wrote, I don't think that Python should workaround OS bugs. If
the OS monotonic clock is not monotonic, the OS should be fixed.

> I can't imagine a scenario where you would ask for a
> monotonic clock and would rather have an error than have Python fill in
> the gap with an emulation.

Sorry, I don't understand what you mean with "fill in the gap with an
emulation". You would like to implement a monotonic clock based on the
system clock?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Scott Dial]

On 3/28/2012 4:48 AM, Victor Stinner wrote:
>>> Scott> monotonic_clock = always goes forward but can be adjusted
>>> Scott> steady_clock = always goes forward and cannot be adjusted
>>>
>>> I don't know if the monotonic clock should be called time.monotonic() or
>>> time.steady(). The clock speed can be adjusted by NTP, at least on Linux
>>> < 2.6.28. (...)
>>
>> You are right that CLOCK_MONOTONIC can be adjusted, so the Boost
>> implementation is wrong. I'm not sure that CLOCK_MONOTONIC_RAW is right
>> either due to suspend -- there doesn't appear to be a POSIX or Linux
>> clock that is defined that meets the "steady" definition.
> 
> The term "adjusted" should be clarified. A clock can be adjusted by
> setting its counter (e.g. setting the system date and time) or by
> changing temporary its frequency (to go faster or slower). Linux only
> adjusts CLOCK_MONOTONIC frequency but the clock is monotonic because
> it always goes forward. The monotonic property can be described as:
> 
> t1=time.monotonic()
> t2=time.monotonic()
> assert t2 >= t1

I agree. The point I was making is that implication of "steady" is that
(t2-t1) is the same (given that t2 and t1 occur in time at the same
relative moments), which is a guarantee that I don't see any platform
providing currently. Any clock that can be "adjusted" in any manner is
not going to meet the "steady" criterion.

>> In that case, I don't think time.try_monotonic() is really needed
>> because we can emulate "time.monotonic()" in software if the platform is
>> deficient.
> 
> As I wrote, I don't think that Python should workaround OS bugs. If
> the OS monotonic clock is not monotonic, the OS should be fixed.

I sympathize with this, but if the idea is that the Python stdlib should
use time.monotonic() for scheduling, then it needs to always be
available. Otherwise, we are not going to use it ourselves, and what
sort of example is that to set?

>> I can't imagine a scenario where you would ask for a
>> monotonic clock and would rather have an error than have Python fill in
>> the gap with an emulation.
> 
> Sorry, I don't understand what you mean with "fill in the gap with an
> emulation". You would like to implement a monotonic clock based on the
> system clock?

If "time.monotonic()" is only sometimes available, then I don't see the
added clock being anything more than an amusement. (In this case, I'd
rather just use clock_gettime() and friends directly, because I have to
be platform aware anyways.) What developers want is a timer that is
useful for scheduling things to happen after predictable interval in the
future, so we should give them that to the best of our ability.

-- 
Scott Dial
sc...@scottdial.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Bug in generator if the generator in created in a C thread

[Victor Stinner]

Hi,

bugs.python.org is down so I'm reporting the bug here :-)

We have a crash in our product when tracing is enabled by
sys.settrace() and threading.settrace(). If a Python generator is
created in a C thread, calling the generator later in another thread
may crash if Python tracing is enabled.

 - the C thread calls PyGILState_Ensure() which creates a temporary
Python thread state
 - a generator is created, the generator has a reference to a Python
frame which keeps a reference to the temporary Python thread state
 - the C thread calls PyGILState_Releases() which destroys the
temporary Python thread state
 - when the generator is called later in another thread, call_trace()
reads the Python thread state from the generator frame, which is the
destroyed frame => it does crash on a pointer dereference if the
memory was reallocated (by malloc()) and the data were erased

To reproduce the crash, unpack the attached
generator_frame_bug.tar.gz, compile the C module using "python
setup.py build" and then run "PYTHONPATH=$(ls -d build/lib*/) python
test.py" (or just "python test.py if you installed the _test module).
You may need to use Valgrind to see the error, or call memset(tstate,
0xFF, sizeof(*tstate)) before free(tstate) in tstate_delete_common().

Calling the generator should update its reference to the Python state
thread in its frame. The generator may also clears frame->f_tstate (to
detect bugs earlier), as it does for frame->f_back (to avoid a
reference cycle). Attached patch implements this fix for Python 3.3.

Victor


generator_frame_bug.tar.gz
Description: GNU Zip compressed data
diff -r 51016ff7f8c9 Objects/genobject.c
--- a/Objects/genobject.c	Sun Mar 25 22:41:16 2012 -0400
+++ b/Objects/genobject.c	Wed Mar 28 12:44:07 2012 +0200
@@ -76,6 +76,7 @@ gen_send_ex(PyGenObject *gen, PyObject *
 
 /* Generators always return to their most recent caller, not
  * necessarily their creator. */
+f->f_tstate = tstate;
 Py_XINCREF(tstate->frame);
 assert(f->f_back == NULL);
 f->f_back = tstate->frame;
@@ -89,6 +90,8 @@ gen_send_ex(PyGenObject *gen, PyObject *
  * cycle. */
 assert(f->f_back == tstate->frame);
 Py_CLEAR(f->f_back);
+/* Clear the borrowed reference to the thread state */
+f->f_tstate = NULL;
 
 /* If the generator just returned (as opposed to yielding), signal
  * that the generator is exhausted. */
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Victor Stinner]

>>> In that case, I don't think time.try_monotonic() is really needed
>>> because we can emulate "time.monotonic()" in software if the platform is
>>> deficient.
>>
>> As I wrote, I don't think that Python should workaround OS bugs. If
>> the OS monotonic clock is not monotonic, the OS should be fixed.
>
> I sympathize with this, but if the idea is that the Python stdlib should
> use time.monotonic() for scheduling, then it needs to always be
> available. Otherwise, we are not going to use it ourselves, and what
> sort of example is that to set?

There is time.hires() if you need a monotonic clock with a fallback to
the system clock.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Steven D'Aprano]

Georg Brandl wrote:

On 28.03.2012 06:45, Nick Coghlan wrote:

On Wed, Mar 28, 2012 at 10:36 AM, Victor Stinner
 wrote:

If QueryPerformanceCounter() is monotonic, the API can be simplified to:

 * time.time() = system clock
 * time.monotonic() = monotonic clock
 * time.hires() = monotonic clock or fallback to system clock

time.hires() definition is exactly what I was trying to implement with
"time.steady(strict=True)" / "time.try_monotonic()".

Please don't call the fallback version "hires" as it suggests it may
be higher resolution than time.time() and that's completely the wrong
idea.


It's also a completely ugly name, since it's quite hard to figure out
what it is supposed to stand for in the first place.


Precisely. I always read "hires" as the verb hires (as in "he hires a car to 
go on holiday") rather than HIgh RESolution.


-1 on hires, it's a horrible name. And misleading as well, because on Linux, 
it isn't any more high res than time.time().


+1 on Nick's suggestion of try_monotonic. It is clear and obvious and doesn't 
mislead.


I don't have an opinion as to what the implementation of try_monotonic should 
be. Whether it should fall back to time.time, time.clock, or something else, I 
don't know. But it is a clear and obvious solution for the use-case of "I 
prefer the monotonic clock, if it is available, otherwise I'll take my chances 
with a best-effect clock."




--
Steven

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[R. David Murray]

On Wed, 28 Mar 2012 23:05:59 +1100, Steven D'Aprano  wrote:
> +1 on Nick's suggestion of try_monotonic. It is clear and obvious and doesn't 
> mislead.

How about "monotonicest".

(No, this is not really a serious suggestion.)

However, time.steadiest might actually work.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Larry Hastings]

On 03/28/2012 01:56 PM, R. David Murray wrote:

On Wed, 28 Mar 2012 23:05:59 +1100, Steven D'Aprano  wrote:

+1 on Nick's suggestion of try_monotonic. It is clear and obvious and doesn't
mislead.

How about "monotonicest".

(No, this is not really a serious suggestion.)


"monotonish".

Thus honoring the Principle Of Least Monotonishment,


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Matt Joiner]

time.monotonic(): The uneventful and colorless function.
On Mar 28, 2012 9:30 PM, "Larry Hastings"  wrote:

> On 03/28/2012 01:56 PM, R. David Murray wrote:
>
>> On Wed, 28 Mar 2012 23:05:59 +1100, Steven D'Aprano
>>  wrote:
>>
>>> +1 on Nick's suggestion of try_monotonic. It is clear and obvious and
>>> doesn't
>>> mislead.
>>>
>> How about "monotonicest".
>>
>> (No, this is not really a serious suggestion.)
>>
>
> "monotonish".
>
> Thus honoring the Principle Of Least Monotonishment,
>
>
> //arry/
> __**_
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> anacrolix%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Jonathan French]

No, that would be time.monotonous(). This is time.monotonic(), the function
that can only play a single note at a time. Uh, I mean time.monophonic().

Hmm, this is harder than it looks.

On 28 March 2012 14:48, Matt Joiner  wrote:

> time.monotonic(): The uneventful and colorless function.
> On Mar 28, 2012 9:30 PM, "Larry Hastings"  wrote:
>
>> On 03/28/2012 01:56 PM, R. David Murray wrote:
>>
>>> On Wed, 28 Mar 2012 23:05:59 +1100, Steven D'Aprano
>>>  wrote:
>>>
 +1 on Nick's suggestion of try_monotonic. It is clear and obvious and
 doesn't
 mislead.

>>> How about "monotonicest".
>>>
>>> (No, this is not really a serious suggestion.)
>>>
>>
>> "monotonish".
>>
>> Thus honoring the Principle Of Least Monotonishment,
>>
>>
>> //arry/
>> __**_
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/**mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
>> anacrolix%40gmail.com
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/me%40jonathanfrench.net
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Guido van Rossum]

Victor,

I have completely lost track of the details of this discussion. Could
you (with help from others who contributed) try to compile a table
showing, for each platform (Windows/Mac/Linux/BSD) which clocks (or
variations) we are considering, and for each of those:

- a link for the reference documentation
- what their typical accuracy is (barring jumps)
- what they do when the "civil" time is made to jump (forward or back)
by the user
- how they are affected by small tweaks to the civil time by NTP
- what they do if the system is suspended and resumed
- whether they can be shared between processes running on the same machine
- whether they may fail or be unsupported under some circumstances

I have a feeling that if I saw such a table it would be much easier to decide.

I assume much of this has already been said at one point in this
thread, but it's impossible to have an overview at the moment.

If someone has more questions they'd like to see answered please add
to the list.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Nick Coghlan]

On Wed, Mar 28, 2012 at 8:56 PM, Victor Stinner
 wrote:
 In that case, I don't think time.try_monotonic() is really needed
 because we can emulate "time.monotonic()" in software if the platform is
 deficient.
>>>
>>> As I wrote, I don't think that Python should workaround OS bugs. If
>>> the OS monotonic clock is not monotonic, the OS should be fixed.
>>
>> I sympathize with this, but if the idea is that the Python stdlib should
>> use time.monotonic() for scheduling, then it needs to always be
>> available. Otherwise, we are not going to use it ourselves, and what
>> sort of example is that to set?
>
> There is time.hires() if you need a monotonic clock with a fallback to
> the system clock.

Completely unintuitive and unnecessary. With the GIL taking care of
synchronisation issues, we can easily coerce time.time() into being a
monotonic clock by the simple expedient of saving the last returned
value:

  def _make_monotic:
  try:
  # Use underlying system monotonic clock if we can
  return _monotonic
  except NameError:
  _tick = time()
  def monotic():
  _new_tick = time()
  if _new_tick > _tick:
  _tick = _new_tick
  return _tick

  monotonic = _make_monotonic()

Monotonicity of the result is thus ensured, even when using
time.time() as a fallback.

If using the system monotonic clock to get greater precision is
acceptable for an application, then forcing monotonicity shouldn't be
a problem either.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Yury Selivanov]

On 2012-03-28, at 10:17 AM, Nick Coghlan wrote:

>  def _make_monotic:
>  try:
>  # Use underlying system monotonic clock if we can
>  return _monotonic
>  except NameError:
>  _tick = time()
>  def monotic():
>  _new_tick = time()
>  if _new_tick > _tick:
>  _tick = _new_tick
>  return _tick
> 
>  monotonic = _make_monotonic()
> 
> Monotonicity of the result is thus ensured, even when using
> time.time() as a fallback.

What if system time jumps 1 year back?  We'll have the same 
monotonic time returned for this whole year?

I don't think we should even try to emulate any of OS-level
functionality.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Guido van Rossum]

On Wed, Mar 28, 2012 at 7:17 AM, Nick Coghlan  wrote:
> On Wed, Mar 28, 2012 at 8:56 PM, Victor Stinner
>  wrote:
> In that case, I don't think time.try_monotonic() is really needed
> because we can emulate "time.monotonic()" in software if the platform is
> deficient.

 As I wrote, I don't think that Python should workaround OS bugs. If
 the OS monotonic clock is not monotonic, the OS should be fixed.
>>>
>>> I sympathize with this, but if the idea is that the Python stdlib should
>>> use time.monotonic() for scheduling, then it needs to always be
>>> available. Otherwise, we are not going to use it ourselves, and what
>>> sort of example is that to set?
>>
>> There is time.hires() if you need a monotonic clock with a fallback to
>> the system clock.
>
> Completely unintuitive and unnecessary. With the GIL taking care of
> synchronisation issues, we can easily coerce time.time() into being a
> monotonic clock by the simple expedient of saving the last returned
> value:
>
>  def _make_monotic:
>      try:
>          # Use underlying system monotonic clock if we can
>          return _monotonic
>      except NameError:
>          _tick = time()
>          def monotic():
>              _new_tick = time()
>              if _new_tick > _tick:
>                  _tick = _new_tick
>              return _tick
>
>  monotonic = _make_monotonic()
>
> Monotonicity of the result is thus ensured, even when using
> time.time() as a fallback.
>
> If using the system monotonic clock to get greater precision is
> acceptable for an application, then forcing monotonicity shouldn't be
> a problem either.

That's a pretty obvious trick. But why don't the kernels do this if
monotonicity is so important? I'm sure there are also downsides, e.g.
if the clock is accidentally set forward by an hour and then back
again, you wouldn't have a useful clock for an hour. And the cache is
not shared between processes so different processes wouldn't see the
same clock value (I presume that most of these clocks have state in
the kernel that isn't bound to any particular process -- AFAIK only
clock() does that, and only on Unixy systems).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Nick Coghlan]

On Wed, Mar 28, 2012 at 6:40 PM, Victor Stinner
 wrote:
>> If we're simplifying the idea to only promising a monotonic
>> clock (i.e. will never go backwards within a given process, but may
>> produce the same value for an indefinite period, and may jump forwards
>> by arbitrarily large amounts),
>
> I don't know any monotonic clock jumping "forwards by arbitrarily
> large amounts". Linux can change CLOCK_MONOTONIC speed, but NTP
> doesn't "jump".

If I understood Glyph's explanation correctly, then if your
application is running in a VM and the VM is getting its clock data
from the underlying hypervisor, then suspending and resuming the VM
may result in forward jumping of the monotonic clocks in the guest OS.
I believe suspending and hibernating may cause similar problems for
even a non-virtualised OS that is getting its time data from a
real-time clock chip that keeps running even when the main CPU goes to
sleep. (If I *misunderstood* Glyph's explanation, then he may have
only been talking about the latter case)

Monotonicity is fairly easy to guarantee - you just remember the last
value you returned and ensure you never return a lower value than that
for the lifetime of the process. The only complication is thread
synchronisation, and the GIL (or a dedicated lock for
Jython/IronPython) can deal with that. Steadiness, on the other hand,
requires a real world time reference and is thus really the domain of
specialised hardware like atomic clocks and GPS units rather than
software that can be suspended and resumed later without changing its
internal state. There's a reason comms station operators pay
substantial chunks of money for time & frequency reference devices
[1].

This is why I now think we only need one new clock function:
time.monotonic(). It will be the system monotonic clock if one is
available, otherwise it will be our own equivalent wrapper around
time.time() that just caches the last value returned to ensure the
result never goes backwards.

With time.monotonic() guaranteed to always be available, there's no
need for a separate function that falls back to an unconditioned
time.time() result.

Regards,
Nick.

[1] For example:
http://www.symmetricom.com/products/gps-solutions/gps-time-frequency-receivers/XLi/

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Guido van Rossum]

On Wed, Mar 28, 2012 at 7:36 AM, Nick Coghlan  wrote:
> On Wed, Mar 28, 2012 at 6:40 PM, Victor Stinner
>  wrote:
>>> If we're simplifying the idea to only promising a monotonic
>>> clock (i.e. will never go backwards within a given process, but may
>>> produce the same value for an indefinite period, and may jump forwards
>>> by arbitrarily large amounts),
>>
>> I don't know any monotonic clock jumping "forwards by arbitrarily
>> large amounts". Linux can change CLOCK_MONOTONIC speed, but NTP
>> doesn't "jump".
>
> If I understood Glyph's explanation correctly, then if your
> application is running in a VM and the VM is getting its clock data
> from the underlying hypervisor, then suspending and resuming the VM
> may result in forward jumping of the monotonic clocks in the guest OS.
> I believe suspending and hibernating may cause similar problems for
> even a non-virtualised OS that is getting its time data from a
> real-time clock chip that keeps running even when the main CPU goes to
> sleep. (If I *misunderstood* Glyph's explanation, then he may have
> only been talking about the latter case)
>
> Monotonicity is fairly easy to guarantee - you just remember the last
> value you returned and ensure you never return a lower value than that
> for the lifetime of the process. The only complication is thread
> synchronisation, and the GIL (or a dedicated lock for
> Jython/IronPython) can deal with that. Steadiness, on the other hand,
> requires a real world time reference and is thus really the domain of
> specialised hardware like atomic clocks and GPS units rather than
> software that can be suspended and resumed later without changing its
> internal state. There's a reason comms station operators pay
> substantial chunks of money for time & frequency reference devices
> [1].
>
> This is why I now think we only need one new clock function:
> time.monotonic(). It will be the system monotonic clock if one is
> available, otherwise it will be our own equivalent wrapper around
> time.time() that just caches the last value returned to ensure the
> result never goes backwards.

As I said, I think the caching idea is bad. We may have to settle for
semantics that are less than perfect -- presumably if you are doing
benchmarking you just have to throw away a bad result that happened to
be affected by a clock anomaly, and if you are using timeouts, retries
are already part of life.

> With time.monotonic() guaranteed to always be available, there's no
> need for a separate function that falls back to an unconditioned
> time.time() result.

I would love to have only one new clock function in 3.3.

> Regards,
> Nick.
>
> [1] For example:
> http://www.symmetricom.com/products/gps-solutions/gps-time-frequency-receivers/XLi/

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bug in generator if the generator in created in a C thread

[Guido van Rossum]

Interesting bug. :-(

It seems bugs.python.org is back up, so can you file it there too?

On Wed, Mar 28, 2012 at 3:45 AM, Victor Stinner
 wrote:
> Hi,
>
> bugs.python.org is down so I'm reporting the bug here :-)
>
> We have a crash in our product when tracing is enabled by
> sys.settrace() and threading.settrace(). If a Python generator is
> created in a C thread, calling the generator later in another thread
> may crash if Python tracing is enabled.
>
>  - the C thread calls PyGILState_Ensure() which creates a temporary
> Python thread state
>  - a generator is created, the generator has a reference to a Python
> frame which keeps a reference to the temporary Python thread state
>  - the C thread calls PyGILState_Releases() which destroys the
> temporary Python thread state
>  - when the generator is called later in another thread, call_trace()
> reads the Python thread state from the generator frame, which is the
> destroyed frame => it does crash on a pointer dereference if the
> memory was reallocated (by malloc()) and the data were erased
>
> To reproduce the crash, unpack the attached
> generator_frame_bug.tar.gz, compile the C module using "python
> setup.py build" and then run "PYTHONPATH=$(ls -d build/lib*/) python
> test.py" (or just "python test.py if you installed the _test module).
> You may need to use Valgrind to see the error, or call memset(tstate,
> 0xFF, sizeof(*tstate)) before free(tstate) in tstate_delete_common().
>
> Calling the generator should update its reference to the Python state
> thread in its frame. The generator may also clears frame->f_tstate (to
> detect bugs earlier), as it does for frame->f_back (to avoid a
> reference cycle). Attached patch implements this fix for Python 3.3.
>
> Victor
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Nick Coghlan]

On Thu, Mar 29, 2012 at 12:27 AM, Yury Selivanov
 wrote:
> What if system time jumps 1 year back?  We'll have the same
> monotonic time returned for this whole year?
>
> I don't think we should even try to emulate any of OS-level
> functionality.

You have to keep in mind the alternative here: falling back to an
*unconditioned* time.time() value (which is the status quo, and
necessary to preserve backwards compatibility). That will break just
as badly in that scenario and is precisely the reason that the OS
level monotonic functionality is desirable in the first place.

I'd be quite happy with a solution that made the OS level monotonic
clock part of the public API, with the caveat that it may not be
available. Then the necessary trio of functions would be:

time.time(): existing system clock, always available
time.os_monotonic(): OS level monotonic clock, not always available
time.monotonic(): always available, same as os_monotonic if it exists,
otherwise uses a time() based emulation that may not be consistent
across processes and may "mark time" for extended periods if the
underlying OS clock is forced to jump back a long way.

I think that naming scheme is more elegant than using monotonic() for
the OS level monotonicity and try_monotonic() for the fallback
version, but I'd be OK with the latter approach, too.

Regards,
Nick.



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Yury Selivanov]

On 2012-03-28, at 10:36 AM, Nick Coghlan wrote:

> Monotonicity is fairly easy to guarantee - you just remember the last
> value you returned and ensure you never return a lower value than that
> for the lifetime of the process.

As I said in my previous mail - I don't think we should ever do that.

Time may jump back and forth, and with your approach it will result in
monotonic() being completely unusable.  If time jumps back for N minutes,
or years, that leads to completely broken expectations for timeouts for N 
minutes or years correspondingly (and that's just the timeouts case, I'm
sure that there are much more critical time-related use-cases.)

If monotonic() will utilize such hack, you add nothing usable in stdlib.
Every serious framework or library will have to re-implement it using
only OS-level functions, and *FAIL* if the OS doesn't support monotonic
time.  Fail, because such framework can't guarantee that it will work 
correctly.

So I think time module should have only one new function: monotonic(),
and this function should be only available if OS provides the underlying
functionality.

No need for steady(), try_monotonic() and other hacks.  Each module
can decide if its dependancy on monotonic is critical or not, and
if it is not, you can always have:

  try:
from time import monotonic as _time
  except ImportError:
from time import time as _time

That's how lots of code is written these days, like using 'epoll' if 
available, and fallback to 'select' if not.  Why don't you try to
abstract differences between them in the standard library?  So I see 
no point in adding some loose abstractions to the stdlib now.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Yury Selivanov]

On 2012-03-28, at 10:45 AM, Nick Coghlan wrote:

> On Thu, Mar 29, 2012 at 12:27 AM, Yury Selivanov
>  wrote:
>> What if system time jumps 1 year back?  We'll have the same
>> monotonic time returned for this whole year?
>> 
>> I don't think we should even try to emulate any of OS-level
>> functionality.
> 
> You have to keep in mind the alternative here: falling back to an
> *unconditioned* time.time() value (which is the status quo, and
> necessary to preserve backwards compatibility). That will break just
> as badly in that scenario and is precisely the reason that the OS
> level monotonic functionality is desirable in the first place.

Well, my argumentation is that you either have some code that depends
on monotonic time and can't work without it, or you have a code that
can work with any time (and only precision matters).  Maybe I'm wrong.

> I'd be quite happy with a solution that made the OS level monotonic
> clock part of the public API, with the caveat that it may not be
> available. Then the necessary trio of functions would be:
> 
> time.time(): existing system clock, always available
> time.os_monotonic(): OS level monotonic clock, not always available
> time.monotonic(): always available, same as os_monotonic if it exists,
> otherwise uses a time() based emulation that may not be consistent
> across processes and may "mark time" for extended periods if the
> underlying OS clock is forced to jump back a long way.

I still don't like this 'emulation' idea.  Smells bad for standard lib.
Big -1 on this approach.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Nick Coghlan]

On Thu, Mar 29, 2012 at 12:42 AM, Guido van Rossum  wrote:
> As I said, I think the caching idea is bad. We may have to settle for
> semantics that are less than perfect -- presumably if you are doing
> benchmarking you just have to throw away a bad result that happened to
> be affected by a clock anomaly, and if you are using timeouts, retries
> are already part of life.

I agree caching doesn't solve the problems that are solved by an OS
level monotonic clock, but falling back to an unmodifided time.time()
result instead doesn't solve those problems either. Falling back to
time.time() just gives you the status quo: time may jump forwards or
backwards by an arbitrary amount between any two calls. Cached
monotonicity just changes the anomalous modes to be time jumping
forwards, or time standing still for an extended period of time. The
only thing the caching provides is that it becomes a reasonable
fallback for a function called time.monotonic() - it *is* a monotonic
clock that meets the formal contract of the function, it's just
nowhere near as good or effective as one the OS can provide.

Forward jumping anomalies aren't as harmful, are very hard to detect
in the first place and behave the same regardless of the presence of
caching, so the interesting case to look at is the difference in
failure modes when the system clock jumps backwards.

For benchmarking, a caching clock will produce a zero result instead
of a negative result. Zeros aren't quite as obviously broken as
negative numbers when benchmarking, but they're still sufficiently
suspicious that most benchmarking activities will flag them as
anomalous. If the jump back was sufficiently small that the subsequent
call still produces a higher value than the original call, then
behaviour reverts to being identical.

For timeouts, setting the clock back means your operation will take
longer to time out than you expected. This problem will occur
regardless of whether you were using cached monotonicity (such that
time stands still) or the system clock (such that time actually goes
backwards). In either case, your deadline will never be reached until
the backwards jump has been cancelled out by the subsequent passage of
time.

I want the standard library to be able to replace its time.time()
calls with time.monotonic(). The only way we can do that without
breaking cross-platform compatibility is if time.monotonic() is
guaranteed to exist, even when the platform only provides time.time().
A dumb caching fallback implementation based on time.time() is the
easiest way to achieve that withou making a complete mockery of the
"monotonic()" name.

There is then a *different* use case, which is 3.3+ only code which
wants to fail noisily when there's no OS level monotonic support - the
application developer really does want to fail *immediately* if
there's no OS level monotonic clock available, instead of crossing
your fingers and hoping you don't hit a clock adjustment glitch
(crossing your fingers has, I'll point out, been the *only* option for
all previous versions of Python, so it clearly can't be *that* scary a
prospect).

So, rather than making time.monotonic() something that the *standard
library can't use*, I'd prefer to address that second use case by
exposing the OS level monotonic clock as time.os_monotonic() only when
it's available. That way, the natural transition for old time.time()
based code is to time.monotonic() (with no cross-platform support
implications), but time.os_monotonic() also becomes available for the
stricter use cases.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Nick Coghlan]

On Thu, Mar 29, 2012 at 1:02 AM, Yury Selivanov  wrote:
> On 2012-03-28, at 10:45 AM, Nick Coghlan wrote:
>
>> On Thu, Mar 29, 2012 at 12:27 AM, Yury Selivanov
>>  wrote:
>>> What if system time jumps 1 year back?  We'll have the same
>>> monotonic time returned for this whole year?
>>>
>>> I don't think we should even try to emulate any of OS-level
>>> functionality.
>>
>> You have to keep in mind the alternative here: falling back to an
>> *unconditioned* time.time() value (which is the status quo, and
>> necessary to preserve backwards compatibility). That will break just
>> as badly in that scenario and is precisely the reason that the OS
>> level monotonic functionality is desirable in the first place.
>
> Well, my argumentation is that you either have some code that depends
> on monotonic time and can't work without it, or you have a code that
> can work with any time (and only precision matters).  Maybe I'm wrong.

You're wrong.

The primary use case for the new time.monotonic() function is to
replace *existing* uses of time.time() in the standard library (mostly
related to timeouts) that are currently vulnerable to clock adjustment
related bugs. This real, concrete use case has been lost in some of
the abstract theoretical discussions that have been going on this
thread.

We can't lose sight of the fact that using a system clock that is
vulnerable to clock adjustment bugs to handle timeouts and
benchmarking in Python has worked just fine for 20+ years. Using a
monotonic clock instead is *better*, but it's far from essential,
since clock adjustments that are big enough and poorly timed enough to
cause real problems are fortunately a very rare occurrence.

So, the primary use case is that we want to replace many of the
time.time() calls in the standard library with time.monotonic() calls.
To avoid backwards compatibility problems in the cross-platform
support, that means time.monotonic() *must be available on every
platform that currently provides time.time()*.

This is why Victor's original proposal was that time.monotonic()
simply fall back to time.time() if there was no OS level monotonic
clock available. The intended use cases are using time.time() *right
now* and have been doing so for years, so it is clearly an acceptable
fallback for those cases. People (rightly, in my opinion) objected to
the idea of time.monotonic() failing to guarantee monotonicity, thus
the proposal to enforce at least a basic level of monotonicity through
caching of the last returned value. I agree completely that this dumb
caching solution doesn't solve any of the original problems with
time.time() that make a time.monotonic() function desirable, but it
isn't meant to. It's only meant to provide graceful degradation to
something that is *no worse than the current behaviour when using
time.time() in Python 3.2* while still respecting the property of
monotonicity for the new API. Yes, it's an ugly hack, but it is a
necessary fallback to avoid accidental regressions in our
cross-platform support.

For the major platforms (i.e. *nix, Mac OS X, Windows), there *will*
be an OS level monotonic clock available, thus using time.monotonic()
will have the desired effect of protecting from clocks being adjusted
backwards. For other platforms, the behaviour (and vulnerabilities)
will be essentially unchanged from the Python 3.2 approach (i.e. using
time.time() with no monotonicity guarantees at all).

However, some 3.3+ applications may want to be stricter about their
behaviour and either bail out completely or fall back to an unfiltered
time.time() call if an OS-level monotonic clock is not available. For
those, it makes sense to expose time.os_monotonic() directly (and only
if it is available), thus allowing those developers to make up their
own mind instead of accepting the cross-platform fallback in
time.monotonic().

Yes, you can get the exact same effect with the "monotonic()" and
"try_monotonic()" naming scheme, but why force the standard library
(and anyone else wanting to upgrade from time.time() without harming
cross-platform support) to use such an ugly name when the
"os_monotonic" and "monotonic" naming scheme provides a much neater
alternative?

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bug in generator if the generator in created in a C thread

[Victor Stinner]

2012/3/28 Guido van Rossum :
> Interesting bug. :-(
>
> It seems bugs.python.org is back up, so can you file it there too?

It took us weeks to track the bug. Here is the issue:
http://bugs.python.org/issue14432

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Guido van Rossum]

On Wed, Mar 28, 2012 at 8:08 AM, Nick Coghlan  wrote:
> On Thu, Mar 29, 2012 at 12:42 AM, Guido van Rossum  wrote:
>> As I said, I think the caching idea is bad. We may have to settle for
>> semantics that are less than perfect -- presumably if you are doing
>> benchmarking you just have to throw away a bad result that happened to
>> be affected by a clock anomaly, and if you are using timeouts, retries
>> are already part of life.
>
> I agree caching doesn't solve the problems that are solved by an OS
> level monotonic clock, but falling back to an unmodifided time.time()
> result instead doesn't solve those problems either. Falling back to
> time.time() just gives you the status quo: time may jump forwards or
> backwards by an arbitrary amount between any two calls. Cached
> monotonicity just changes the anomalous modes to be time jumping
> forwards, or time standing still for an extended period of time. The
> only thing the caching provides is that it becomes a reasonable
> fallback for a function called time.monotonic() - it *is* a monotonic
> clock that meets the formal contract of the function, it's just
> nowhere near as good or effective as one the OS can provide.

TBH, I don't think like this focus on monotonicity as the most
important feature.

> Forward jumping anomalies aren't as harmful, are very hard to detect
> in the first place and behave the same regardless of the presence of
> caching, so the interesting case to look at is the difference in
> failure modes when the system clock jumps backwards.

Agreed.

> For benchmarking, a caching clock will produce a zero result instead
> of a negative result. Zeros aren't quite as obviously broken as
> negative numbers when benchmarking, but they're still sufficiently
> suspicious that most benchmarking activities will flag them as
> anomalous. If the jump back was sufficiently small that the subsequent
> call still produces a higher value than the original call, then
> behaviour reverts to being identical.

So for benchmarking we don't care about jumps, really, and the caching
version is slightly less useful.

> For timeouts, setting the clock back means your operation will take
> longer to time out than you expected. This problem will occur
> regardless of whether you were using cached monotonicity (such that
> time stands still) or the system clock (such that time actually goes
> backwards). In either case, your deadline will never be reached until
> the backwards jump has been cancelled out by the subsequent passage of
> time.

Where in the stdlib do we actually calculate timeouts instead of using
the timeouts built into the OS (e.g. select())?

I think it would be nice if we could somehow use the *same* clock as
the OS uses to implement timeouts.

> I want the standard library to be able to replace its time.time()
> calls with time.monotonic().

Where in the stdlib? (I'm aware of threading.py. Any other places?)

> The only way we can do that without
> breaking cross-platform compatibility is if time.monotonic() is
> guaranteed to exist, even when the platform only provides time.time().
> A dumb caching fallback implementation based on time.time() is the
> easiest way to achieve that withou making a complete mockery of the
> "monotonic()" name.

Yeah, so maybe it's a bad name. :-)

> There is then a *different* use case, which is 3.3+ only code which
> wants to fail noisily when there's no OS level monotonic support - the
> application developer really does want to fail *immediately* if
> there's no OS level monotonic clock available, instead of crossing
> your fingers and hoping you don't hit a clock adjustment glitch
> (crossing your fingers has, I'll point out, been the *only* option for
> all previous versions of Python, so it clearly can't be *that* scary a
> prospect).
>
> So, rather than making time.monotonic() something that the *standard
> library can't use*, I'd prefer to address that second use case by
> exposing the OS level monotonic clock as time.os_monotonic() only when
> it's available. That way, the natural transition for old time.time()
> based code is to time.monotonic() (with no cross-platform support
> implications), but time.os_monotonic() also becomes available for the
> stricter use cases.

I'd be happier if the fallback function didn't try to guarantee things
the underlying clock can't guarantee. I.e. I like the idea of having a
function that uses some accurate OS clock if one exists but falls back
to time.time() if not; I don't like the idea of that new function
trying to interpret the value of time.time() in any way. Applications
that need the OS clock's guarantees can call it directly. We could
also offer something where you can introspect the properties of the
clock (or clocks) so that an app can choose the best clock depending
on its needs.

To summarize my problem with the caching idea: take a simple timeout
loop such as found in several places in threading.py.

def wait_for(delta, eps):
  # Wait for delta seconds

Re: [Python-Dev] PEP 418: Add monotonic clock

[Matt Joiner]

time.timeout_clock?

Everyone knows what that will be for and we won't have to make silly
theoretical claims about its properties and expected uses.

If no one else looks before I next get to a PC I'll dig up the clock/timing
source used for select and friends, and find any corresponding syscall that
retrieves it for Linux.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Nick Coghlan]

On Thu, Mar 29, 2012 at 1:47 AM, Guido van Rossum  wrote:
> Where in the stdlib? (I'm aware of threading.py. Any other places?)

Victor had at least one other example. multiprocessing, maybe? I
believe the test suite may still have a few instances as well.

> But now consider a caching clock, and consider that the system clock
> made a jump backwards *before* this function is called. The cache
> prevents us from seeing it, so the initial call to now() returns the
> highest clock value seen so far. And until the system clock has caught
> up with that, now() will return the same value over and over -- so WE
> STILL WAIT TOO  LONG.

Ouch. OK, I'm convinced the caching fallback is worse than just
falling back to time.time() directly, which means the naming problem
needs to be handled another way.

> My conclusion: you can't win this game by forcing the clock to return
> a monotonic value. A better approach might be to compute how many
> sleep(eps) calls we're expected to make, and to limit the loop to that
> -- although sleep() doesn't make any guarantees either about sleeping
> too short or too long. Basically, if you do sleep(1) and find that
> your clock didn't move (enough), you can't tell the difference between
> a short sleep and a clock that jumped back. And if your clock moved to
> much, you still don't know if the problem was with sleep() or with
> your clock.

With your point about the problem with the naive caching mechanism
acknowledged, I think we can safely assign time.monotonic() as the
name of the OS provided monotonic clock.

That means choosing a name for the version that falls back to time()
if monotonic() isn't available so it can be safely substituted for
time.time() without having to worry about platform compatibility
implications. I don't like Victor's current "hires" (because it
doesn't hint at the fallback behaviour, may actually be the same res
as time.time() and reads like an unrelated English word). My own
suggestion of "try_monotic()" would get the job done, but is hardly
going to win any API beauty contests.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Steven D'Aprano]

Matt Joiner wrote:

time.timeout_clock?

Everyone knows what that will be for and we won't have to make silly
theoretical claims about its properties and expected uses.


I don't.



--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Stephen J. Turnbull]

On Thu, Mar 29, 2012 at 1:14 AM, Nick Coghlan  wrote:
> That means choosing a name for the version that falls back to time()
> if monotonic() isn't available so it can be safely substituted for
> time.time() without having to worry about platform compatibility
> implications.

What's wrong with "time.time()" again?  As documented in
http://docs.python.org/py3k/library/time.html it makes no guarantees,
and specifically there is *no* guarantee that it will ever behave
*badly*.  Of course, we'll have to guarantee that, if a
badly-behaved clock is available, users can get access to it, so call
that time._time().
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Steven D'Aprano]

Nick Coghlan wrote:


Completely unintuitive and unnecessary. With the GIL taking care of
synchronisation issues, we can easily coerce time.time() into being a
monotonic clock by the simple expedient of saving the last returned
value:

[snip]


Here's a version that doesn't suffer from the flaw of returning a long stream 
of constant values when the system clock jumps backwards a significant amount:


class MockTime:
def __init__(self):
self.ticks = [1, 2, 3, 4, 2, 3, 4, 5, 7, 3, 4, 6, 7, 8, 8, 9]
self.i = -1
def __call__(self):
self.i += 1
return self.ticks[self.i]

time = MockTime()

_prev = _prev_raw = 0
def monotonic():
global _prev, _prev_raw
raw = time()
delta = max(0, raw - _prev_raw)
_prev_raw = raw
_prev += delta
return _prev


And in use:

>>> [monotonic() for i in range(16)]
[1, 2, 3, 4, 4, 5, 6, 7, 9, 9, 10, 12, 13, 14, 14, 15]


Time: [1, 2, 3, 4, 2, 3, 4, 5, 7, 3,  4,  6,  7,  8,  8,  9]
Nick: [1, 2, 3, 4, 4, 4, 4, 5, 7, 7,  7,  7,  7,  8,  8,  9]
Mine: [1, 2, 3, 4, 4, 5, 6, 7, 9, 9, 10, 12, 13, 14, 14, 15]

Mine will get ahead of the system clock each time it jumps back, but it's a 
lot closer to the ideal of a *strictly* monotonically increasing clock.


Assuming that the system clock will never jump backwards twice in a row, the 
double-caching version will never have more than two constant values in a row.




--
Steven

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Lennart Regebro]

On Wed, Mar 28, 2012 at 12:56, Victor Stinner  wrote:
> There is time.hires() if you need a monotonic clock with a fallback to
> the system clock.

Does this primarily give a high resolution clock, or primarily a
monotonic clock? That's not clear from either the name, or the PEP.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Charles-François Natali]

> What's wrong with "time.time()" again?  As documented in
> http://docs.python.org/py3k/library/time.html it makes no guarantees,
> and specifically there is *no* guarantee that it will ever behave
> *badly*.  Of course, we'll have to guarantee that, if a
> badly-behaved clock is available, users can get access to it, so call
> that time._time().

I'm not sure I understand your suggestion correctly, but replacing
time.time() by time.monotonic() with fallback won't work, because
time.monotonic() isn't wall-clock time: it can very well use an
arbitrary reference point (most likely system start-up time).

As for the hires() function, since there's no guarantee whatsoever
that it does provide a better resolution than time.time(), this would
be really misleading IMHO.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bug tracker outage

[Martin v. Löwis]

Am 27.03.2012 23:11, schrieb "Martin v. Löwis":
> Upfront hosting (Izak Burger) is going to do a Debian upgrade of the bug
> tracker machine "soon" (likely tomorrow). This may cause some outage,
> since there is a lot of custom stuff on the machine which may
> break with newer (Python) versions. I'll notify here when the upgrade
> is complete.

The upgrade is complete. It was in fact the Postgres upgrade (to 8.4)
which caused the longest blackout time.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Yury Selivanov]

On 2012-03-28, at 11:35 AM, Nick Coghlan wrote:
> So, the primary use case is that we want to replace many of the
> time.time() calls in the standard library with time.monotonic() calls.
> To avoid backwards compatibility problems in the cross-platform
> support, that means time.monotonic() *must be available on every
> platform that currently provides time.time()*.

OK.  I got your point.  And also I've just realized what I dislike about
the way you want to implement the fallback.  The main problem is that
I treat the situation when time jumps backward as an exception, because,
again, if you have timeouts you may get those timeouts to never be 
executed.  So let's make the "try_monotonic()" function (or whatever 
name will be chosen) this way (your original code edited):

  def _make_monotic():
 try:
 # Use underlying system monotonic clock if we can
 return _monotonic
 except NameError:
 _tick = time()
 def monotic():
 nonlocal _time
 _new_tick = time()
 if _new_tick <= _tick:
 raise RuntimeError('time was adjusted backward')
 _tick = _new_tick
 return _new_tick

 return monotonic

  try_monotonic = _make_monotonic()  

At least this approach tries to follow some of the python's zen.

-
Yury

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Virtualenv not portable from Python 2.7.2 to 2.7.3 (os.urandom missing)

[Jason R. Coombs]

I see this was reported as a debian bug.
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=665776

 

We encountered it as well.

 

To reproduce, using virtualenv 1.7+ on Python 2.7.2 on Ubuntu, create a
virtualenv. Move that virtualenv to a host with Python 2.7.3RC2 yields:

 

jaraco@vdm-dev:~$ /usr/bin/python2.7 -V

Python 2.7.3rc2

jaraco@vdm-dev:~$ env/bin/python -V

Python 2.7.2

jaraco@vdm-dev:~$ env/bin/python -c "import os; os.urandom()"

Traceback (most recent call last):

  File "", line 1, in 

AttributeError: 'module' object has no attribute 'urandom'

 

This bug causes Django to not start properly (under some circumstances).

 

I reviewed the changes between v2.7.2 and 2.7 (tip) and it seems there was
substantial refactoring of the os and posix modules for urandom.

 

I still don't fully understand why the urandom method is missing (because
the env includes the python 2.7.2 executable and stdlib).

 

I suspect this change is going to cause some significant backward
compatibility issues. Is there a recommended workaround? Should I file a
bug?

 

Regards,

Jason



smime.p7s
Description: S/MIME cryptographic signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Virtualenv not portable from Python 2.7.2 to 2.7.3 (os.urandom missing)

[Carl Meyer]

Hi Jason,

On 03/28/2012 12:22 PM, Jason R. Coombs wrote:
> To reproduce, using virtualenv 1.7+ on Python 2.7.2 on Ubuntu, create a
> virtualenv. Move that virtualenv to a host with Python 2.7.3RC2 yields:
> 
> jaraco@vdm-dev:~$ /usr/bin/python2.7 -V
> 
> Python 2.7.3rc2
> 
> jaraco@vdm-dev:~$ env/bin/python -V
> 
> Python 2.7.2
> 
> jaraco@vdm-dev:~$ env/bin/python -c "import os; os.urandom()"
> 
> Traceback (most recent call last):
> 
>   File "", line 1, in 
> 
> AttributeError: 'module' object has no attribute 'urandom'
> 
> This bug causes Django to not start properly (under some circumstances).
> 
> I reviewed the changes between v2.7.2 and 2.7 (tip) and it seems there
> was substantial refactoring of the os and posix modules for urandom.
> 
> I still don’t fully understand why the urandom method is missing
> (because the env includes the python 2.7.2 executable and stdlib).

In Python 2.6.8/2.7.3, urandom is built into the executable. A
virtualenv doesn't contain the whole stdlib, only the bits necessary to
bootstrap site.py. So the problem arises from trying to use the 2.7.3
stdlib with a 2.7.2 interpreter.

> I suspect this change is going to cause some significant backward
> compatibility issues. Is there a recommended workaround? Should I file a
> bug?

The workaround is easy: just re-run virtualenv on that path with the new
interpreter.

I was made aware of this issue a few weeks ago, and added a warning to
the virtualenv "news" page:
http://www.virtualenv.org/en/latest/news.html  I'm not sure where else
to publicize it.

Carl



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Yury Selivanov]

On 2012-03-28, at 1:55 PM, Yury Selivanov wrote:

> nonlocal _time

nonlocal _tick

obviously.

P.S. And we can make it to raise an error after some N calls
to time() resulting in lesser time that is stored in the _tick
variable.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] datetime module and pytz with dateutil

[Andrew Svetlov]

I figured out what pytz and dateutil are not mentioned in python docs
for datetime module.
It's clean why these libs is not a part of Python Libraries — but
that's not clean for Docs.
From my perspective at least pytz (as py3k compatible) should to be
mentioned as the library which contains timezone info, supported
carefully and recommended to use with datetime standard module,

-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Virtualenv not portable from Python 2.7.2 to 2.7.3 (os.urandom missing)

[Jason R. Coombs]

> -Original Message-
> From: python-dev-bounces+jaraco=jaraco@python.org [mailto:python-
> dev-bounces+jaraco=jaraco@python.org] On Behalf Of Carl Meyer
> Sent: Wednesday, 28 March, 2012 14:48
> 
> The workaround is easy: just re-run virtualenv on that path with the new
> interpreter.
> 

Thanks for the quick response Carl. I appreciate all the work that's been
done.

I'm not sure the workaround is as simple as you say. Virtualenv doesn't
replace the 'python' exe if it already exists (because it may already exist
for a different minor version of Python (3.2, 2.6)). So the procedure is
probably something like this:

For each  version of Python the virtualenv wraps (ls
env/bin/python?.?):
1) Run env/bin/python -V. If the result starts with "Python ", remove
env/bin/python.
2) Determine if that Python version uses distribute or setuptools.
3) Run virtualenv --python=python env (with --distribute if
appropriate)

I haven't yet tested this procedure, but I believe it's closer to what will
need to be done. There are probably other factors. Unfortunately, to
reliably repair the virtualenv is very difficult, so we will probably opt
with re-deploying all of our virtualenvs.

Will the release notes include something about this change, since it will
likely have broad backward incompatibility for all existing virtualenvs? I
wouldn't expect someone in operations to read the virtualenv news to find
out what things a Python upgrade will break. Indeed, this update will
probably be pushed out as part of standard, unattended system updates.

I realize that the relationship between stdlib.os and posixmodule isn't a
guaranteed interface, and the fact that it breaks with virtualenv is a
weakness of virtualenv. Nevertheless, virtualenv has become the defacto
technique for Python environments. Putting my sysops cap on, I might
perceive this change as being unannounced (w.r.t. Python) and having
significant impact on operations. I would think this impact deserves at
least a note in the release notes.

Regards,
Jason


smime.p7s
Description: S/MIME cryptographic signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime module and pytz with dateutil

[Guido van Rossum]

+1 If pytz is py3k cabable. -1 for dateutIl.

On Wednesday, March 28, 2012, Andrew Svetlov wrote:

> I figured out what pytz and dateutil are not mentioned in python docs
> for datetime module.
> It's clean why these libs is not a part of Python Libraries — but
> that's not clean for Docs.
> From my perspective at least pytz (as py3k compatible) should to be
> mentioned as the library which contains timezone info, supported
> carefully and recommended to use with datetime standard module,
>
> --
> Thanks,
> Andrew Svetlov
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Victor Stinner]

> I would love to have only one new clock function in 3.3.

I already added time.clock_gettime() to 3.3 :-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Scott Dial]

On 3/28/2012 10:29 AM, Guido van Rossum wrote:
> On Wed, Mar 28, 2012 at 7:17 AM, Nick Coghlan  wrote:
>> Completely unintuitive and unnecessary. With the GIL taking care of
>> synchronisation issues, we can easily coerce time.time() into being a
>> monotonic clock by the simple expedient of saving the last returned
>> value:
> 
> That's a pretty obvious trick. But why don't the kernels do this if
> monotonicity is so important? I'm sure there are also downsides, e.g.
> if the clock is accidentally set forward by an hour and then back
> again, you wouldn't have a useful clock for an hour. And the cache is
> not shared between processes so different processes wouldn't see the
> same clock value (I presume that most of these clocks have state in
> the kernel that isn't bound to any particular process -- AFAIK only
> clock() does that, and only on Unixy systems).
> 

What makes you think that isn't already true? I don't know what
platforms that CPython compiles for that *won't* have one of the
aforementioned functions available that provide a *real* monotonic
clock. Surely, any platform that doesn't didn't recognize the need for
it, or they would just provide a monotonic clock. That is to say, if you
are a POSIX compliant system, then there is no reason to break
gettimeofday() and friends when you can just implement CLOCK_MONOTONIC
proper (even if it's just a trick like Nick's).

I think the PEP should enumerate what platforms that CPython supports
that will not benefit from a real monotonic clock. I think the number of
platforms will be such a minority that the emulation makes sense.
Practicality beats purity, and all.

-- 
Scott Dial
sc...@scottdial.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Victor Stinner]

> Does this primarily give a high resolution clock, or primarily a
> monotonic clock? That's not clear from either the name, or the PEP.

I expect a better resolution from time.monotonic() than time.time(). I
don't have exact numbers right now, but I began to document each OS
clock in the PEP.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime module and pytz with dateutil

[Andrew Svetlov]

I'm personally +1 for pytz only — dateutil is big enough and...
Well, can we just point to pytz in our docs for datetime module?


On Thu, Mar 29, 2012 at 12:06 AM, Guido van Rossum  wrote:
> +1 If pytz is py3k cabable. -1 for dateutIl.
>
>
> On Wednesday, March 28, 2012, Andrew Svetlov wrote:
>>
>> I figured out what pytz and dateutil are not mentioned in python docs
>> for datetime module.
>> It's clean why these libs is not a part of Python Libraries — but
>> that's not clean for Docs.
>> From my perspective at least pytz (as py3k compatible) should to be
>> mentioned as the library which contains timezone info, supported
>> carefully and recommended to use with datetime standard module,
>>
>> --
>> Thanks,
>> Andrew Svetlov
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
> --
> --Guido van Rossum (python.org/~guido)



-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Victor Stinner]

> I think the PEP should enumerate what platforms that CPython supports
> that will not benefit from a real monotonic clock. I think the number of
> platforms will be such a minority that the emulation makes sense.
> Practicality beats purity, and all.

The PEP lists OS monotonic clocks by platform. Windows, Mac OS X,
Solaris, and "UNIX" (CLOCK_MONOTONIC & friends) provide monotonic
clocks. I don't know any platform without monotonic clock.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Virtualenv not portable from Python 2.7.2 to 2.7.3 (os.urandom missing)

[R. David Murray]

On Wed, 28 Mar 2012 20:56:30 -, "Jason R. Coombs"  wrote:
> Will the release notes include something about this change, since it will
> likely have broad backward incompatibility for all existing virtualenvs? I
> wouldn't expect someone in operations to read the virtualenv news to find
> out what things a Python upgrade will break. Indeed, this update will
> probably be pushed out as part of standard, unattended system updates.

I think it is reasonable to put something in the release notes.  This
change is much larger than changes we normally make in maintenance
release, because it fixes a security bug.  But because it is larger
than normal, adding release notes like this about known breakage is, I
think, a good idea.

Perhaps you and Carl could collaborate on a page explaining the issue in
detail, and on a brief note to include in the release notes that points
to your more extensive discussion?

But keep in mind I'm not the release manager; we'll need their buy
in on this.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Guido van Rossum]

On Wed, Mar 28, 2012 at 2:36 PM, Scott Dial
 wrote:
> On 3/28/2012 10:29 AM, Guido van Rossum wrote:
>> On Wed, Mar 28, 2012 at 7:17 AM, Nick Coghlan  wrote:
>>> Completely unintuitive and unnecessary. With the GIL taking care of
>>> synchronisation issues, we can easily coerce time.time() into being a
>>> monotonic clock by the simple expedient of saving the last returned
>>> value:
>>
>> That's a pretty obvious trick. But why don't the kernels do this if
>> monotonicity is so important? I'm sure there are also downsides, e.g.
>> if the clock is accidentally set forward by an hour and then back
>> again, you wouldn't have a useful clock for an hour. And the cache is
>> not shared between processes so different processes wouldn't see the
>> same clock value (I presume that most of these clocks have state in
>> the kernel that isn't bound to any particular process -- AFAIK only
>> clock() does that, and only on Unixy systems).
>>
>
> What makes you think that isn't already true?

What does "that" refer to in this sentence?


> I don't know what
> platforms that CPython compiles for that *won't* have one of the
> aforementioned functions available that provide a *real* monotonic
> clock. Surely, any platform that doesn't didn't recognize the need for
> it, or they would just provide a monotonic clock. That is to say, if you
> are a POSIX compliant system, then there is no reason to break
> gettimeofday() and friends when you can just implement CLOCK_MONOTONIC
> proper (even if it's just a trick like Nick's).
>
> I think the PEP should enumerate what platforms that CPython supports
> that will not benefit from a real monotonic clock. I think the number of
> platforms will be such a minority that the emulation makes sense.
> Practicality beats purity, and all.
>
> --
> Scott Dial
> sc...@scottdial.com



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] bug tracker offline again for re-indexing

[R. David Murray]

Since Martin hasn't sent a note about this here I will:

I noticed that text search wasn't working right on the bug tracker, and Martin
has taken it offline again to re-index.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bug tracker offline again for re-indexing

[Martin v. Löwis]

Am 28.03.2012 23:55, schrieb R. David Murray:
> Since Martin hasn't sent a note about this here I will:
> 
> I noticed that text search wasn't working right on the bug tracker, and Martin
> has taken it offline again to re-index.

which will, unfortunately, take a few more hours to complete.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Victor Stinner]

> Where in the stdlib do we actually calculate timeouts instead of using
> the timeouts built into the OS (e.g. select())?

At least in threading and queue modules. The common use case is to
retry a function with a timeout if the syscall was interrupted by a
signal (EINTR error). The socket module and _threading.Lock.acquire()
implement such retry loop using the system clock. They should use a
monotonic clock instead.

> I think it would be nice if we could somehow use the *same* clock as
> the OS uses to implement timeouts.

On Linux, nanosleep() uses CLOCK_MONOTONIC whereas POSIX suggests
CLOCK_REALTIME. Some functions allow to choose the clock, like pthread
locks or clock_nanosleep().

> I'd be happier if the fallback function didn't try to guarantee things
> the underlying clock can't guarantee. I.e. I like the idea of having a
> function that uses some accurate OS clock if one exists but falls back
> to time.time() if not; I don't like the idea of that new function
> trying to interpret the value of time.time() in any way.

We may workaround some OS known bugs like:
http://support.microsoft.com/?id=274323

The link contains an example how to workaround the bug. The idea of
the workaround is to use two different monotonic clocks to detect
leaps, with one trusted clock (GetTickCount) and one untrusted clock
having an higher resolution (QueryPerformanceCounter).

I don't think that the same algorithm is applicable on other OSes
because other OSes usually only provide one monotonic clock, sometimes
though different API.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic clock

[Victor Stinner]

> Could you (with help from others who contributed) try to compile a table
> showing, for each platform (Windows/Mac/Linux/BSD) which clocks (or
> variations) we are considering, and for each of those:
>
> - a link for the reference documentation
> - what their typical accuracy is (barring jumps)
> - what they do when the "civil" time is made to jump (forward or back)
> by the user
> - how they are affected by small tweaks to the civil time by NTP
> - what they do if the system is suspended and resumed
> - whether they can be shared between processes running on the same machine
> - whether they may fail or be unsupported under some circumstances
>
> I have a feeling that if I saw such a table it would be much easier to decide.
>
> I assume much of this has already been said at one point in this
> thread, but it's impossible to have an overview at the moment.

I don't know where I can get all these information, but I'm completing
the PEP each time that I find a new information. It's difficult to get
the accuracy of a clock and how it handles system suspend.

I'm intereted if anyone has such information.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bug tracker offline again for re-indexing

[Martin v. Löwis]

>> I noticed that text search wasn't working right on the bug tracker, and 
>> Martin
>> has taken it offline again to re-index.
> 
> which will, unfortunately, take a few more hours to complete.

It seems to work now, so I turned it on again. Text search now uses
Xapian, and recreating the Xapian index of all msg objects took a while.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com