Re: [Python-Dev] Python and compilers

[Willian Sodré da Paixão]

Thank you all. My decision at the moment is a compiler for PICs to
accept a language more like python as possible instead of C.
How is a college project "solo", I can guarantee at least the first
part, a lexical analyzer.

Ass: Wil ('.')
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

[Nir Aides]

Hi Peter,

There is no need for a loop in bfs_yield().


On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante
wrote:

>  Nir,
>
> Per the POSIX standard, both pthread_cond_wait() and
> pthread_cond_timedwait() need to be performed in a loop.  See the fourth
> paragraph of the description from:
>
>
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html
>
>
> For the Windows side, I think you have a similar problem. Condition
> variables are signaling mechanisms, and so they have a separate boolean
> predicate associated with them. If you release the mutex that protects the
> predicate, then after you reacquire the mutex, you have to reevaluate the
> predicate to ensure that the condition has actually been met.
>
> You might want to look at the following for a discussion (not sure how good
> it is, as I just google’d it quickly) of how to implement POSIX semantics on
> Windows:
>
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html
>
>
> Before you can evaluate the effectiveness of any of the proposed scheduling
> schemes, the fundamental uses of mutexes and condition variables, and their
> implementations, must be sound.
>
> -peter
>
>
>
> On 4/11/10 6:50 PM, "Nir Aides" < n...@winpdb.org> wrote:
>
> Hello all,
>
> I would like to kick this discussion back to life with a simplified
> implementation of the BFS scheduler, designed by the Linux kernel hacker Con
> Kolivas: http://ck.kolivas.org/patches/bfs/sched-BFS.txt
>
> I submitted bfs.patch at  http://bugs.python.org/issue7946. It is work in
> progress but is ready for some opinion.
>
> On my machine BFS gives comparable performance to gilinter, and seems to
> schedule threads more fairly, predictably, and with lower rate of context
> switching. Its basic design is very simple but nevertheless it was designed
> by an expert in this field, two characteristics which combine to make it
> attractive to this case.
>
> The problem addressed by the GIL has always been *scheduling* threads to
> the interpreter, not just controlling access to it, and therefore the GIL, a
> lock implemented as a simple semaphore was the wrong solution.
>
> The patches by Antoine and David essentially evolve the GIL into a
> scheduler, however both cause thread starvation or high rate of context
> switching in some scenarios:
>
> With Floren't write test ( http://bugs.python.org/issue7946#msg101120):
> 2 bg threads, 2 cores set to performance, karmic, PyCon patch, context
> switching shoots up to 200K/s.
> 2 bg threads, 1 core, set to on-demand, karmic, idle machine, gilinter
> patch starves one of the bg threads.
> 4 bg threads, 4x1 core xeon, centos 5.3, gilinter patch, all bg threads
> starved, context switching shoots up to 250K/s.
>
> With UDP test ( http://bugs.python.org/file16316/udp-iotest.py), add
> zlib.compress(b'GIL') to the workload:
> both gilinter and PyCon patches starve the IO thread.
>
> The BFS patch currently involves more overhead by reading the time stamp on
> each yield and schedule operations. In addition it still remains to address
> some issues related to timestamps such as getting different time stamp
> readings on different cores on some (older) multi-core systems.
>
> Any thoughts?
>
> Nir
>
>
>
> On Sun, Mar 14, 2010 at 12:46 AM, Antoine Pitrou < solip...@pitrou.net>
> wrote:
>
>
> Hello,
>
> As some of you may know, Dave Beazley recently exhibited a situation
> where the new GIL shows quite a poor behaviour (the old GIL isn't very
> good either, but still a little better). This issue is followed in
> http://bugs.python.org/issue7946
>
> This situation is when an IO-bound thread wants to process a lot of
> incoming packets, while one (or several) CPU-bound thread is also
> running. Each time the IO-bound thread releases the GIL, the CPU-bound
> thread gets it and keeps holding it for at least 5 milliseconds
> (default setting), which limits the number of individual packets which
> can be recv()'ed and processed per second.
>
> I have proposed two mechanisms, based on the same idea: IO-bound
> threads should be able to steal the GIL very quickly, rather than
> having to wait for the whole "thread switching interval" (again, 5 ms
> by default). They differ in how they detect an "IO-bound threads":
>
> - the first mechanism is actually the same mechanism which was
>   embodied in the original new GIL patch before being removed. In this
>   approach, IO methods (such as socket.read() in socketmodule.c)
>   releasing the GIL must use a separate C macro when trying to get the
>   GIL back again.
>
> - the second mechanism dynamically computes the "interactiveness" of a
>   thread and allows interactive threads to steal the GIL quickly. In
>   this approach, IO methods don't have to be modified at all.
>
> Both approaches show similar benchmark results (for the benchmarks
> that I know of) and basically fix the issue put forward by Dave Beazley.
>
> Any thoughts?
>
> Regards
>
> Antoine.
>
>
> __ ___

Re: [Python-Dev] "Fixing" the new GIL

[Peter Portante]

Hmm, so I see in bfs_yield():

+if (tstate != NULL && bfs_thread_switch == tstate) {
+COND_RESET(tstate->bfs_cond);
+COND_WAIT(tstate->bfs_cond, bfs_mutex);
+}

So, to me, the above code says, ³Wait for the condition that tstate is
either NULL, or bfs_thread_switch does not equal tstate². So the predicate
is: ³(tstate != NULL && bfs_thread_switch == tstate)².

If the predicate is True before you call COND_WAIT() and True after you call
COND_WAIT(), either you don¹t need to call COND_WAIT() at all, or you need
to loop until the predicate is False. There is no guarantee that a condition
wait actually did anything at all. Yes, there will be spurious wake ups, but
you can¹t tell if the thread actually blocked and then woke up, or never
blocked at all. If it never actually blocks, then that code path is not
helpful.

On Windows, before this loop in bfs_schedule():

+COND_RESET(tstate->bfs_cond);
+while (bfs_thread != tstate) {
+_bfs_timed_wait(tstate, timestamp);
+timestamp = get_timestamp();
+}

You might want to avoid the call to reset the condition variable if the
predicate is already False.

-peter


On 4/12/10 8:12 AM, "Nir Aides"  wrote:

> Hi Peter,
> 
> There is no need for a loop in bfs_yield(). 
> 
> 
> On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante 
> wrote:
>> Nir,
>> 
>> Per the POSIX standard, both pthread_cond_wait() and pthread_cond_timedwait()
>> need to be performed in a loop.  See the fourth paragraph of the description
>> from:
>> 
>>> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwa
>>> it.html 
>>> >> ait.html> 
>> 
>> For the Windows side, I think you have a similar problem. Condition variables
>> are signaling mechanisms, and so they have a separate boolean predicate
>> associated with them. If you release the mutex that protects the predicate,
>> then after you reacquire the mutex, you have to reevaluate the predicate to
>> ensure that the condition has actually been met.
>> 
>> You might want to look at the following for a discussion (not sure how good
>> it is, as I just google¹d it quickly) of how to implement POSIX semantics on
>> Windows:
>> 
>>> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html
>>> 
>> 
>> Before you can evaluate the effectiveness of any of the proposed scheduling
>> schemes, the fundamental uses of mutexes and condition variables, and their
>> implementations, must be sound.
>> 
>> -peter
>> 
>> 
>> 
>> On 4/11/10 6:50 PM, "Nir Aides" < n...@winpdb.org  >
>> wrote:
>> 
>>> Hello all,
>>> 
>>> I would like to kick this discussion back to life with a simplified
>>> implementation of the BFS scheduler, designed by the Linux kernel hacker Con
>>> Kolivas: http://ck.kolivas.org/patches/bfs/sched-BFS.txt
>>> 
>>> 
>>> I submitted bfs.patch at  http://bugs.python.org/issue7946
>>>  . It is work in progress but is ready for
>>> some opinion.
>>> 
>>> On my machine BFS gives comparable performance to gilinter, and seems to
>>> schedule threads more fairly, predictably, and with lower rate of context
>>> switching. Its basic design is very simple but nevertheless it was designed
>>> by an expert in this field, two characteristics which combine to make it
>>> attractive to this case.
>>> 
>>> The problem addressed by the GIL has always been *scheduling* threads to the
>>> interpreter, not just controlling access to it, and therefore the GIL, a
>>> lock implemented as a simple semaphore was the wrong solution.
>>> 
>>> The patches by Antoine and David essentially evolve the GIL into a
>>> scheduler, however both cause thread starvation or high rate of context
>>> switching in some scenarios:
>>> 
>>> With Floren't write test ( http://bugs.python.org/issue7946#msg101120
>>>  ):
>>> 2 bg threads, 2 cores set to performance, karmic, PyCon patch, context
>>> switching shoots up to 200K/s.
>>> 2 bg threads, 1 core, set to on-demand, karmic, idle machine, gilinter patch
>>> starves one of the bg threads.
>>> 4 bg threads, 4x1 core xeon, centos 5.3, gilinter patch, all bg threads
>>> starved, context switching shoots up to 250K/s.
>>> 
>>> With UDP test ( http://bugs.python.org/file16316/udp-iotest.py
>>>  ), add
>>> zlib.compress(b'GIL') to the workload:
>>> both gilinter and PyCon patches starve the IO thread.
>>> 
>>> The BFS patch currently involves more overhead by reading the time stamp on
>>> each yield and schedule operations. In addition it still remains to address
>>> some issues related to timestamps such as getting different time stamp
>>> readings on different cores on some (older) multi-core systems.
>>> 
>>> Any thoughts?
>>> 
>>> Nir
>>> 
>>> 
>>>

Re: [Python-Dev] "Fixing" the new GIL

[Peter Portante]

And why the for(;;) loop in bfs_schedule()? I don¹t see a code path that
would loop there. Perhaps I am missing it ...

-peter


On 4/12/10 8:37 AM, "Peter Portante"  wrote:

> Hmm, so I see in bfs_yield():
> 
> +if (tstate != NULL && bfs_thread_switch == tstate) {
> +COND_RESET(tstate->bfs_cond);
> +COND_WAIT(tstate->bfs_cond, bfs_mutex);
> +}
> 
> So, to me, the above code says, ³Wait for the condition that tstate is either
> NULL, or bfs_thread_switch does not equal tstate². So the predicate is:
> ³(tstate != NULL && bfs_thread_switch == tstate)².
> 
> If the predicate is True before you call COND_WAIT() and True after you call
> COND_WAIT(), either you don¹t need to call COND_WAIT() at all, or you need to
> loop until the predicate is False. There is no guarantee that a condition wait
> actually did anything at all. Yes, there will be spurious wake ups, but you
> can¹t tell if the thread actually blocked and then woke up, or never blocked
> at all. If it never actually blocks, then that code path is not helpful.
> 
> On Windows, before this loop in bfs_schedule():
> 
> +COND_RESET(tstate->bfs_cond);
> +while (bfs_thread != tstate) {
> +_bfs_timed_wait(tstate, timestamp);
> +timestamp = get_timestamp();
> +}
> 
> You might want to avoid the call to reset the condition variable if the
> predicate is already False.
> 
> -peter
> 
> 
> On 4/12/10 8:12 AM, "Nir Aides"  wrote:
> 
>> Hi Peter,
>> 
>> There is no need for a loop in bfs_yield(). 
>> 
>> 
>> On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante 
>> wrote:
>>> Nir,
>>> 
>>> Per the POSIX standard, both pthread_cond_wait() and
>>> pthread_cond_timedwait() need to be performed in a loop.  See the fourth
>>> paragraph of the description from:
>>> 
 http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedw
 ait.html 
  
>>> 
>>> For the Windows side, I think you have a similar problem. Condition
>>> variables are signaling mechanisms, and so they have a separate boolean
>>> predicate associated with them. If you release the mutex that protects the
>>> predicate, then after you reacquire the mutex, you have to reevaluate the
>>> predicate to ensure that the condition has actually been met.
>>> 
>>> You might want to look at the following for a discussion (not sure how good
>>> it is, as I just google¹d it quickly) of how to implement POSIX semantics on
>>> Windows:
>>> 
 http://www.cs.wustl.edu/~schmidt/win32-cv-1.html
 
>>> 
>>> Before you can evaluate the effectiveness of any of the proposed scheduling
>>> schemes, the fundamental uses of mutexes and condition variables, and their
>>> implementations, must be sound.
>>> 
>>> -peter
>>> 
>>> 
>>> 
>>> On 4/11/10 6:50 PM, "Nir Aides" < n...@winpdb.org  >
>>> wrote:
>>> 
 Hello all,
 
 I would like to kick this discussion back to life with a simplified
 implementation of the BFS scheduler, designed by the Linux kernel hacker
 Con Kolivas: http://ck.kolivas.org/patches/bfs/sched-BFS.txt
 
 
 I submitted bfs.patch at  http://bugs.python.org/issue7946
  . It is work in progress but is ready
 for some opinion.
 
 On my machine BFS gives comparable performance to gilinter, and seems to
 schedule threads more fairly, predictably, and with lower rate of context
 switching. Its basic design is very simple but nevertheless it was designed
 by an expert in this field, two characteristics which combine to make it
 attractive to this case.
 
 The problem addressed by the GIL has always been *scheduling* threads to
 the interpreter, not just controlling access to it, and therefore the GIL,
 a lock implemented as a simple semaphore was the wrong solution.
 
 The patches by Antoine and David essentially evolve the GIL into a
 scheduler, however both cause thread starvation or high rate of context
 switching in some scenarios:
 
 With Floren't write test ( http://bugs.python.org/issue7946#msg101120
  ):
 2 bg threads, 2 cores set to performance, karmic, PyCon patch, context
 switching shoots up to 200K/s.
 2 bg threads, 1 core, set to on-demand, karmic, idle machine, gilinter
 patch starves one of the bg threads.
 4 bg threads, 4x1 core xeon, centos 5.3, gilinter patch, all bg threads
 starved, context switching shoots up to 250K/s.
 
 With UDP test ( http://bugs.python.org/file16316/udp-iotest.py
  ), add
 zlib.compress(b'GIL') to the workload:
 both gilinter and PyCon patches starve the IO thread.
 
 The BFS patch currently

Re: [Python-Dev] [RELEASED] 2.7 beta 1

[anatoly techtonik]

On Sun, Apr 11, 2010 at 1:13 AM, average  wrote:
>
> There are so many features taken from 3.0 that I fear that it will
> postpone its adoption interminably (it is, in practice, treated as
> "beta" software itself).  By making it doctrine that it won't be
> official until the next "major" Python release, it will encourage
> those who are able, to just make the jump to 3.0, while those who
> cannot will have the subtle pressure to make the shift, however
> gradual.

> Additionally, it will give the community further incentive
> to make Python3 all that it was intended to be.  Personally, the
> timing of v3 prevented me from fully participating in that effort,
> and, not ignoring the work of those who did contribute, I think many
> of us feel that it has not reached its potential.

The same problem. For me it was possible to participate in standard
library development only after Python Alphas with Windows binaries
were released. I could test both new features and old bugs. Having a
requirement that every developer should be able to compile binaries
has an adverse effect on the quality of standard library.

The absence of public Roadmap also makes it hard to judge the
aforementioned "desired potential".
It could be possible to compile a public list like
http://dungeonhack.sourceforge.net/Roadmap

I am afraid of two things with forthcoming Python releases.
1. feature creeping
2. feature missing
And an overview of Python development in the form of release timer and
roadmap will remove the remnants of fear and uncertainty and surely
attract new people for sprints.

Regardless of said above it is great to feel the hard work behind the
scenes that makes new releases popping up. Thanks.
-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] OS information, tags

[Anand Balachandran Pillai]

Hi,

   I am surprised to see that the bug-tracker
doesn't have an OS classifier or ability to add
tags ? Since a number of issues reported seem to
be OS specific (one can find a lot of Windows only
issues upon a search), won't adding these fields
help bug triaging ?

I am not sure which software is being used by
bug tracker so excuse me if this has been already
discussed in this form. A quick search in my gmail
archives yielded no such discussion.

Thanks,

-- 
--Anand
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] OS information, tags

[Senthil Kumaran]

On Mon, Apr 12, 2010 at 07:06:29PM +0530, Anand Balachandran Pillai wrote:
>    I am surprised to see that the bug-tracker
> doesn't have an OS classifier or ability to add
> tags ? Since a number of issues reported seem to

There is one. In the Components you can do a multiple select and it
has Macintosh , Windows as options.

> I am not sure which software is being used by
> bug tracker so excuse me if this has been already
> discussed in this form. A quick search in my gmail
> archives yielded no such discussion.

It is Roundup developed by Richard (Pygame ??). There is  tracker bug
list too, which you can find referenced in the bugs.python.org page.


-- 
Senthil

"Elves and Dragons!" I says to him.  "Cabbages and potatoes are better
for you and me."
-- J. R. R. Tolkien
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] OS information, tags

[R. David Murray]

On Mon, 12 Apr 2010 19:06:29 +0530, Anand Balachandran Pillai 
 wrote:
>I am surprised to see that the bug-tracker
> doesn't have an OS classifier or ability to add
> tags ? Since a number of issues reported seem to
> be OS specific (one can find a lot of Windows only
> issues upon a search), won't adding these fields
> help bug triaging ?

As someone else pointed out, we do have Windows and Mac components.
Historically the number of bugs specific to other OSes has been small
enough that it hasn't been worth having classifiers for them.  Or perhaps
it's that we haven't had devs who were specifically tracking those
platforms...

We can add additional components if the community sees a need for them.

--
R. David Murray  www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Traceback object has no __class__?

[Terry Reedy]

On 4/11/2010 6:43 AM, Greg Ewing wrote:

import sys

try:
   raise ValueError
except ValueError:
   tb = sys.exc_info()[2]
   print tb
   print tb.__class__


# 3.1.2



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

[Nir Aides]

The loop-less wait is similar to the one in new GIL. It is used to force a
switch to next thread in particular scenario and the motivation is explained
in comment to another if clause a few lines up. Those two if clauses can be
joined though.


On Mon, Apr 12, 2010 at 3:37 PM, Peter Portante
wrote:

>  Hmm, so I see in bfs_yield():
>
> +if (tstate != NULL && bfs_thread_switch == tstate) {
> +COND_RESET(tstate->bfs_cond);
> +COND_WAIT(tstate->bfs_cond, bfs_mutex);
> +}
>
> So, to me, the above code says, “Wait for the condition that tstate is
> either NULL, or bfs_thread_switch does not equal tstate”. So the predicate
> is: “(tstate != NULL && bfs_thread_switch == tstate)”.
>
> If the predicate is True before you call COND_WAIT() and True after you
> call COND_WAIT(), either you don’t need to call COND_WAIT() at all, or you
> need to loop until the predicate is False. There is no guarantee that a
> condition wait actually did anything at all. Yes, there will be spurious
> wake ups, but you can’t tell if the thread actually blocked and then woke
> up, or never blocked at all. If it never actually blocks, then that code
> path is not helpful.
>
> On Windows, before this loop in bfs_schedule():
>
> +COND_RESET(tstate->bfs_cond);
> +while (bfs_thread != tstate) {
> +_bfs_timed_wait(tstate, timestamp);
> +timestamp = get_timestamp();
> +}
>
> You might want to avoid the call to reset the condition variable if the
> predicate is already False.
>
> -peter
>
>
>
> On 4/12/10 8:12 AM, "Nir Aides"  wrote:
>
> Hi Peter,
>
> There is no need for a loop in bfs_yield().
>
>
> On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante <
> peter.a.porta...@gmail.com> wrote:
>
> Nir,
>
> Per the POSIX standard, both pthread_cond_wait() and
> pthread_cond_timedwait() need to be performed in a loop.  See the fourth
> paragraph of the description from:
>
>
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html<
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html>
>
>
>
> For the Windows side, I think you have a similar problem. Condition
> variables are signaling mechanisms, and so they have a separate boolean
> predicate associated with them. If you release the mutex that protects the
> predicate, then after you reacquire the mutex, you have to reevaluate the
> predicate to ensure that the condition has actually been met.
>
> You might want to look at the following for a discussion (not sure how good
> it is, as I just google’d it quickly) of how to implement POSIX semantics on
> Windows:
>
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html <
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html>
>
>
> Before you can evaluate the effectiveness of any of the proposed scheduling
> schemes, the fundamental uses of mutexes and condition variables, and their
> implementations, must be sound.
>
> -peter
>
>
>
> On 4/11/10 6:50 PM, "Nir Aides" <
> n...@winpdb.org  > wrote:
>
> Hello all,
>
> I would like to kick this discussion back to life with a simplified
> implementation of the BFS scheduler, designed by the Linux kernel hacker Con
> Kolivas: http://ck.kolivas.org/patches/bfs/sched-BFS.txt <
> http://ck.kolivas.org/patches/bfs/sched-BFS.txt>
>
> I submitted bfs.patch at  http://bugs.python.org/issue7946 <
> http://bugs.python.org/issue7946> . It is work in progress but is ready
> for some opinion.
>
>
> On my machine BFS gives comparable performance to gilinter, and seems to
> schedule threads more fairly, predictably, and with lower rate of context
> switching. Its basic design is very simple but nevertheless it was designed
> by an expert in this field, two characteristics which combine to make it
> attractive to this case.
>
> The problem addressed by the GIL has always been *scheduling* threads to
> the interpreter, not just controlling access to it, and therefore the GIL, a
> lock implemented as a simple semaphore was the wrong solution.
>
> The patches by Antoine and David essentially evolve the GIL into a
> scheduler, however both cause thread starvation or high rate of context
> switching in some scenarios:
>
> With Floren't write test ( http://bugs.python.org/issue7946#msg101120 <
> http://bugs.python.org/issue7946#msg101120> ):
>
> 2 bg threads, 2 cores set to performance, karmic, PyCon patch, context
> switching shoots up to 200K/s.
> 2 bg threads, 1 core, set to on-demand, karmic, idle machine, gilinter
> patch starves one of the bg threads.
> 4 bg threads, 4x1 core xeon, centos 5.3, gilinter patch, all bg threads
> starved, context switching shoots up to 250K/s.
>
> With UDP test ( http://bugs.python.org/file16316/udp-iotest.py <
> http://bugs.python.org/file16316/udp-iotest.py> ), add
> zlib.compress(b'GIL') to the workload:
>
> both gilinter and PyCon patches starve the IO thread.
>
> The BFS patch currently involves more overhead by reading the time stamp on
> each yie

Re: [Python-Dev] "Fixing" the new GIL

[Nir Aides]

At some point there was a loop, later it remained since I feel it is more
readable than a bunch of nested if-else clauses.
Should probably replace with do {...} while(0)

I was conditioned with electrical shocks in the dungeons of a corporate to
always use for loops.

I uploaded the patch to Rietveld for code review comments:
http://codereview.appspot.com/857049

Nir


On Mon, Apr 12, 2010 at 3:48 PM, Peter Portante
wrote:

>  And why the for(;;) loop in bfs_schedule()? I don’t see a code path that
> would loop there. Perhaps I am missing it ...
>
> -peter
>
>
>
> On 4/12/10 8:37 AM, "Peter Portante"  wrote:
>
> Hmm, so I see in bfs_yield():
>
> +if (tstate != NULL && bfs_thread_switch == tstate) {
> +COND_RESET(tstate->bfs_cond);
> +COND_WAIT(tstate->bfs_cond, bfs_mutex);
> +}
>
> So, to me, the above code says, “Wait for the condition that tstate is
> either NULL, or bfs_thread_switch does not equal tstate”. So the predicate
> is: “(tstate != NULL && bfs_thread_switch == tstate)”.
>
> If the predicate is True before you call COND_WAIT() and True after you
> call COND_WAIT(), either you don’t need to call COND_WAIT() at all, or you
> need to loop until the predicate is False. There is no guarantee that a
> condition wait actually did anything at all. Yes, there will be spurious
> wake ups, but you can’t tell if the thread actually blocked and then woke
> up, or never blocked at all. If it never actually blocks, then that code
> path is not helpful.
>
> On Windows, before this loop in bfs_schedule():
>
> +COND_RESET(tstate->bfs_cond);
> +while (bfs_thread != tstate) {
> +_bfs_timed_wait(tstate, timestamp);
> +timestamp = get_timestamp();
> +}
>
> You might want to avoid the call to reset the condition variable if the
> predicate is already False.
>
> -peter
>
>
> On 4/12/10 8:12 AM, "Nir Aides"  wrote:
>
> Hi Peter,
>
> There is no need for a loop in bfs_yield().
>
>
> On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante <
> peter.a.porta...@gmail.com> wrote:
>
> Nir,
>
> Per the POSIX standard, both pthread_cond_wait() and
> pthread_cond_timedwait() need to be performed in a loop.  See the fourth
> paragraph of the description from:
>
>
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html<
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html>
>
>
>
> For the Windows side, I think you have a similar problem. Condition
> variables are signaling mechanisms, and so they have a separate boolean
> predicate associated with them. If you release the mutex that protects the
> predicate, then after you reacquire the mutex, you have to reevaluate the
> predicate to ensure that the condition has actually been met.
>
> You might want to look at the following for a discussion (not sure how good
> it is, as I just google’d it quickly) of how to implement POSIX semantics on
> Windows:
>
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html <
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html>
>
>
> Before you can evaluate the effectiveness of any of the proposed scheduling
> schemes, the fundamental uses of mutexes and condition variables, and their
> implementations, must be sound.
>
> -peter
>
>
>
> On 4/11/10 6:50 PM, "Nir Aides" < n...@winpdb.org  >
> wrote:
>
> Hello all,
>
> I would like to kick this discussion back to life with a simplified
> implementation of the BFS scheduler, designed by the Linux kernel hacker Con
> Kolivas: http://ck.kolivas.org/patches/bfs/sched-BFS.txt <
> http://ck.kolivas.org/patches/bfs/sched-BFS.txt>
>
> I submitted bfs.patch at  http://bugs.python.org/issue7946 <
> http://bugs.python.org/issue7946> . It is work in progress but is ready
> for some opinion.
>
> On my machine BFS gives comparable performance to gilinter, and seems to
> schedule threads more fairly, predictably, and with lower rate of context
> switching. Its basic design is very simple but nevertheless it was designed
> by an expert in this field, two characteristics which combine to make it
> attractive to this case.
>
> The problem addressed by the GIL has always been *scheduling* threads to
> the interpreter, not just controlling access to it, and therefore the GIL, a
> lock implemented as a simple semaphore was the wrong solution.
>
> The patches by Antoine and David essentially evolve the GIL into a
> scheduler, however both cause thread starvation or high rate of context
> switching in some scenarios:
>
> With Floren't write test ( http://bugs.python.org/issue7946#msg101120 <
> http://bugs.python.org/issue7946#msg101120> ):
> 2 bg threads, 2 cores set to performance, karmic, PyCon patch, context
> switching shoots up to 200K/s.
> 2 bg threads, 1 core, set to on-demand, karmic, idle machine, gilinter
> patch starves one of the bg threads.
> 4 bg threads, 4x1 core xeon, centos 5.3, gilinter patch, all bg threads
> starved, context switching shoots up to 250K/s.
>
> W

Re: [Python-Dev] "Fixing" the new GIL

[Peter Portante]

Yes, but unless you loop until the predicate is False, no forced switch is
guaranteed to occur. You might as well remove the code. If you want to keep
the code as is, call me when you need a life guard to help debug mystifying
behaviors. ;) -peter


On 4/12/10 3:17 PM, "Nir Aides"  wrote:

> The loop-less wait is similar to the one in new GIL. It is used to force a
> switch to next thread in particular scenario and the motivation is explained
> in comment to another if clause a few lines up. Those two if clauses can be
> joined though.
> 
> 
> On Mon, Apr 12, 2010 at 3:37 PM, Peter Portante 
> wrote:
>> Hmm, so I see in bfs_yield():
>> 
>> +    if (tstate != NULL && bfs_thread_switch == tstate) {
>> +    COND_RESET(tstate->bfs_cond);
>> +    COND_WAIT(tstate->bfs_cond, bfs_mutex);
>> +    }
>> 
>> So, to me, the above code says, ³Wait for the condition that tstate is either
>> NULL, or bfs_thread_switch does not equal tstate². So the predicate is:
>> ³(tstate != NULL && bfs_thread_switch == tstate)².
>> 
>> If the predicate is True before you call COND_WAIT() and True after you call
>> COND_WAIT(), either you don¹t need to call COND_WAIT() at all, or you need to
>> loop until the predicate is False. There is no guarantee that a condition
>> wait actually did anything at all. Yes, there will be spurious wake ups, but
>> you can¹t tell if the thread actually blocked and then woke up, or never
>> blocked at all. If it never actually blocks, then that code path is not
>> helpful.
>> 
>> On Windows, before this loop in bfs_schedule():
>> 
>> +    COND_RESET(tstate->bfs_cond);
>> +    while (bfs_thread != tstate) {
>> +    _bfs_timed_wait(tstate, timestamp);
>> +    timestamp = get_timestamp();
>> +    }
>> 
>> You might want to avoid the call to reset the condition variable if the
>> predicate is already False.
>> 
>> -peter
>> 
>> 
>> 
>> On 4/12/10 8:12 AM, "Nir Aides" http://n...@winpdb.org> >
>> wrote:
>> 
>>> Hi Peter,
>>> 
>>> There is no need for a loop in bfs_yield(). 
>>> 
>>> 
>>> On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante >>  > wrote:
 Nir,
 
 Per the POSIX standard, both pthread_cond_wait() and
 pthread_cond_timedwait() need to be performed in a loop.  See the fourth
 paragraph of the description from:
 
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timed
> wait.html 
>  dwait.html> 
 
 For the Windows side, I think you have a similar problem. Condition
 variables are signaling mechanisms, and so they have a separate boolean
 predicate associated with them. If you release the mutex that protects the
 predicate, then after you reacquire the mutex, you have to reevaluate the
 predicate to ensure that the condition has actually been met.
 
 You might want to look at the following for a discussion (not sure how good
 it is, as I just google¹d it quickly) of how to implement POSIX semantics
 on Windows:
 
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html
> 
 
 Before you can evaluate the effectiveness of any of the proposed scheduling
 schemes, the fundamental uses of mutexes and condition variables, and their
 implementations, must be sound.
 
 -peter
 
 
 
 On 4/11/10 6:50 PM, "Nir Aides" <
  n...@winpdb.org    > 
 wrote:
 
> Hello all,
> 
> I would like to kick this discussion back to life with a simplified
> implementation of the BFS scheduler, designed by the Linux kernel hacker
> Con Kolivas: http://ck.kolivas.org/patches/bfs/sched-BFS.txt
> 
> 
> I submitted bfs.patch at  http://bugs.python.org/issue7946
>  . It is work in progress but is ready
> for some opinion.
> 
> 
> On my machine BFS gives comparable performance to gilinter, and seems to
> schedule threads more fairly, predictably, and with lower rate of context
> switching. Its basic design is very simple but nevertheless it was
> designed by an expert in this field, two characteristics which combine to
> make it attractive to this case.
> 
> The problem addressed by the GIL has always been *scheduling* threads to
> the interpreter, not just controlling access to it, and therefore the GIL,
> a lock implemented as a simple semaphore was the wrong solution.
> 
> The patches by Antoine and David essentially evolve the GIL into a
> scheduler, however both cause thread starvation or high rate of context
> switching in some scenarios:
> 
> With Floren't write test ( http://bugs.python.org/issue7946#msg101120
> 

Re: [Python-Dev] "Fixing" the new GIL

[Nir Aides]

Please describe clearly a step by step scenario in which that code will
fail.


On Mon, Apr 12, 2010 at 10:25 PM, Peter Portante  wrote:

>  Yes, but unless you loop until the predicate is False, no forced switch
> is guaranteed to occur. You might as well remove the code. If you want to
> keep the code as is, call me when you need a life guard to help debug
> mystifying behaviors. ;) -peter
>
>
>
> On 4/12/10 3:17 PM, "Nir Aides"  wrote:
>
> The loop-less wait is similar to the one in new GIL. It is used to force a
> switch to next thread in particular scenario and the motivation is explained
> in comment to another if clause a few lines up. Those two if clauses can be
> joined though.
>
>
> On Mon, Apr 12, 2010 at 3:37 PM, Peter Portante <
> peter.a.porta...@gmail.com> wrote:
>
> Hmm, so I see in
> bfs_yield():
>
> +if (tstate != NULL && bfs_thread_switch == tstate) {
> +COND_RESET(tstate->bfs_cond);
> +COND_WAIT(tstate->bfs_cond, bfs_mutex);
> +}
>
> So, to me, the above code says, “Wait for the condition that tstate is
> either NULL, or bfs_thread_switch does not equal tstate”. So the predicate
> is: “(tstate != NULL && bfs_thread_switch == tstate)”.
>
> If the predicate is True before you call COND_WAIT() and True after you
> call COND_WAIT(), either you don’t need to call COND_WAIT() at all, or you
> need to loop until the predicate is False. There is no guarantee that a
> condition wait actually did anything at all. Yes, there will be spurious
> wake ups, but you can’t tell if the thread actually blocked and then woke
> up, or never blocked at all. If it never actually blocks, then that code
> path is not helpful.
>
> On Windows, before this loop in bfs_schedule():
>
> +COND_RESET(tstate->bfs_cond);
> +while (bfs_thread != tstate) {
> +_bfs_timed_wait(tstate, timestamp);
> +timestamp = get_timestamp();
> +}
>
> You might want to avoid the call to reset the condition variable if the
> predicate is already False.
>
> -peter
>
>
>
> On 4/12/10 8:12 AM, "Nir Aides" http://n...@winpdb.org> >
> wrote:
>
> Hi Peter,
>
> There is no need for a loop in bfs_yield().
>
>
> On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante <
> peter.a.porta...@gmail.com  > wrote:
>
> Nir,
>
> Per the POSIX standard, both pthread_cond_wait() and
> pthread_cond_timedwait() need to be performed in a loop.  See the fourth
> paragraph of the description from:
>
>
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html<
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html>
>
>
>
> For the Windows side, I think you have a similar problem. Condition
> variables are signaling mechanisms, and so they have a separate boolean
> predicate associated with them. If you release the mutex that protects the
> predicate, then after you reacquire the mutex, you have to reevaluate the
> predicate to ensure that the condition has actually been met.
>
> You might want to look at the following for a discussion (not sure how good
> it is, as I just google’d it quickly) of how to implement POSIX semantics on
> Windows:
>
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html <
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html>
>
>
> Before you can evaluate the effectiveness of any of the proposed scheduling
> schemes, the fundamental uses of mutexes and condition variables, and their
> implementations, must be sound.
>
> -peter
>
>
>
> On 4/11/10 6:50 PM, "Nir Aides" <
>  n...@winpdb.org   > wrote:
>
> Hello all,
>
> I would like to kick this discussion back to life with a simplified
> implementation of the BFS scheduler, designed by the Linux kernel hacker Con
> Kolivas: http://ck.kolivas.org/patches/bfs/sched-BFS.txt <
> http://ck.kolivas.org/patches/bfs/sched-BFS.txt>
>
> I submitted bfs.patch at  http://bugs.python.org/issue7946 <
> http://bugs.python.org/issue7946> . It is work in progress but is ready
> for some opinion.
>
>
> On my machine BFS gives comparable performance to gilinter, and seems to
> schedule threads more fairly, predictably, and with lower rate of context
> switching. Its basic design is very simple but nevertheless it was designed
> by an expert in this field, two characteristics which combine to make it
> attractive to this case.
>
> The problem addressed by the GIL has always been *scheduling* threads to
> the interpreter, not just controlling access to it, and therefore the GIL, a
> lock implemented as a simple semaphore was the wrong solution.
>
> The patches by Antoine and David essentially evolve the GIL into a
> scheduler, however both cause thread starvation or high rate of context
> switching in some scenarios:
>
> With Floren't write test ( http://bugs.python.org/issue7946#msg101120 <
> http://bugs.python.org/issue7946#msg101120> ):
>
> 2 bg threads, 2 cores set to performance, karmic, PyCon patch, context
> s

Re: [Python-Dev] "Fixing" the new GIL

[Peter Portante]

That code will not switch to another thread if the condition variable either
does not actually block the thread on the call (which is allowed by the
standard to give implementations some flexibility for making it work
correctly ­ read the standard reasoning for more information), or the thread
is woken up without a predicate change (also allowed by the standard for
similar reasons). Both of those cases are called ³spurious wake-ups².

You may not be able to readily get your implementation to behavior that way,
but in the wild, we need to account for this behavior because Cpython will
be run on systems where it will happen. :)

-peter


On 4/12/10 3:36 PM, "Nir Aides"  wrote:

> Please describe clearly a step by step scenario in which that code will fail.
> 
> 
> On Mon, Apr 12, 2010 at 10:25 PM, Peter Portante 
> wrote:
>> Yes, but unless you loop until the predicate is False, no forced switch is
>> guaranteed to occur. You might as well remove the code. If you want to keep
>> the code as is, call me when you need a life guard to help debug mystifying
>> behaviors. ;) -peter
>> 
>> 
>> 
>> On 4/12/10 3:17 PM, "Nir Aides" http://n...@winpdb.org> >
>> wrote:
>> 
>>> The loop-less wait is similar to the one in new GIL. It is used to force a
>>> switch to next thread in particular scenario and the motivation is explained
>>> in comment to another if clause a few lines up. Those two if clauses can be
>>> joined though.
>>> 
>>> 
>>> On Mon, Apr 12, 2010 at 3:37 PM, Peter Portante >>  > wrote:
 Hmm, so I see in
 bfs_yield():
 
 +    if (tstate != NULL && bfs_thread_switch == tstate) {
 +    COND_RESET(tstate->bfs_cond);
 +    COND_WAIT(tstate->bfs_cond, bfs_mutex);
 +    }
 
 So, to me, the above code says, ³Wait for the condition that tstate is
 either NULL, or bfs_thread_switch does not equal tstate². So the predicate
 is: ³(tstate != NULL && bfs_thread_switch == tstate)².
 
 If the predicate is True before you call COND_WAIT() and True after you
 call COND_WAIT(), either you don¹t need to call COND_WAIT() at all, or you
 need to loop until the predicate is False. There is no guarantee that a
 condition wait actually did anything at all. Yes, there will be spurious
 wake ups, but you can¹t tell if the thread actually blocked and then woke
 up, or never blocked at all. If it never actually blocks, then that code
 path is not helpful.
 
 On Windows, before this loop in bfs_schedule():
 
 +    COND_RESET(tstate->bfs_cond);
 +    while (bfs_thread != tstate) {
 +    _bfs_timed_wait(tstate, timestamp);
 +    timestamp = get_timestamp();
 +    }
 
 You might want to avoid the call to reset the condition variable if the
 predicate is already False.
 
 -peter
 
 
 
 On 4/12/10 8:12 AM, "Nir Aides" http://n...@winpdb.org>
  > wrote:
 
> Hi Peter,
> 
> There is no need for a loop in bfs_yield(). 
> 
> 
> On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante
> http://peter.a.porta...@gmail.com>
>  > wrote:
>> Nir,
>> 
>> Per the POSIX standard, both pthread_cond_wait() and
>> pthread_cond_timedwait() need to be performed in a loop.  See the fourth
>> paragraph of the description from:
>> 
>>> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_tim
>>> edwait.html
>>> >> medwait.html>
>> 
>> For the Windows side, I think you have a similar problem. Condition
>> variables are signaling mechanisms, and so they have a separate boolean
>> predicate associated with them. If you release the mutex that protects
>> the predicate, then after you reacquire the mutex, you have to reevaluate
>> the predicate to ensure that the condition has actually been met.
>> 
>> You might want to look at the following for a discussion (not sure how
>> good it is, as I just google¹d it quickly) of how to implement POSIX
>> semantics on Windows:
>> 
>>> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html
>>> 
>> 
>> Before you can evaluate the effectiveness of any of the proposed
>> scheduling schemes, the fundamental uses of mutexes and condition
>> variables, and their implementations, must be sound.
>> 
>> -peter
>> 
>> 
>> 
>> On 4/11/10 6:50 PM, "Nir Aides" <
>>  
>> n...@winpdb.org   
>>  > wrote:
>> 
>>> Hello all,
>>> 
>>> I would like to kick this discussion back to life with a simplified
>>> implementation of the BFS scheduler, designed by the Linux kernel hacker
>>

Re: [Python-Dev] Issue #7978, unexpected EINTR-related exceptions

[Martin (gzlist)]

On 08/04/2010, Victor Stinner  wrote:
> Le jeudi 08 avril 2010 08:11:09, Yaniv Aknin a écrit :
>> Issue #7978 (http://bugs.python.org/issue7978) describes a bug in
>> SocketServer where a received signal in SocketServer's select() call
>> will raise an uncaught exception due to EINTR. The proposed solution
>> was to wrap SocketServer's select() with something like twisted's
>> untilConcludes function, which catches EINTR exceptions and re-calls
>> the call (see issue for code).
>>
>> However, before committing this to SocketServer, neologix raised the
>> (valid) concern that this is generally-useful code, already duplicated
>> at least in subprocess (_eintr_retry_call, subprocess.py), which can
>> probably be moved to another module and be shared from there. (...)
>
> +1 to share EINTR-related code, but I don't know the best place for such
> functions. Maybe the os module?

Bazaar did a similar thing in copying the Twisted function to a
generic module for use throughout the codebase. From the experience
there, I think replicating that would be a bad idea.

There are a very large number of functions in the Python standard
library that can throw EINTR, but hardly any of them can be "fixed" by
wrapping them in a function that catches the error and mindlessly
recalls the function with the same arguments.

In Bazaar, until_no_eintr proved to be an attractive nuisance that
lead to incorrect code. Providing such an easily misused interface in
the standard library is likely to do more harm than good.

There are some errors and other confusions in the initial post and
later, but reading this thread from the Bazaar list should give an
impression:


Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] stdlib socket usage and "keepalive"

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Debugging a strange problem today, I got the following result:

Sockets open by stdlib libraries are open without the "keepalive"
option, so the system default is used. The system default under linux is
"no keepalive".

So, if you are using a URLlib connection, POP3 connection, IMAP
connection, etc., any stdlib that internally creates a socket, and your
server goes away suddendly (you lose network connectivity, by instance),
the library will wait FOREVER for the server. The client can't detect
that the server is not longer available.

The "keepalive" option will send a probe packed every X minutes of
inactivity, to check if the other side is still alive, even if the
connection is idle.

The issue is bad, but the solution is simple enough. Options:

1. All "client" libraries should create sockets with the "KEEPALIVE" option.

2. Modify the socket C module to create all sockets as "Keepalive" by
default.

3. To have a global variable in the socket module to change the default
for future sockets. Something like current "socket.setdefaulttimeout()".
The default should be "keepalive".

4. Modify client libraries to accept a new optional socket-like object
as an optional parameter. This would allow things like transparent
compression or encryption, or to replace the socket connection by
anything else (read/write to shared memory or database, for example).

This is an issue in Linux because by default the sockets are not
"keepalive". In other Unix systems, the default is "keepalive". I don't
know about MS Windows.

What do you think?. The solution seems trivial, after deciding the right
way to go.

PS: "socket.setdefaulttimeout()" is not enough, because it could
shutdown a perfectly functional connection, just because it was idle for
too long.

- -- 
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBS8OV2Zlgi5GaxT1NAQIJhwP8CH+gij4KfrJ1oHW5ys4PboH5Ru0pgdly
Wbsza0+uj3p68P1vDnC9jIr7j+fI3ql3DOc8zUoIKGpJoaWVspbv3c4vI4ATLHo+
J6I18dpkviRT8/sT/69vMvghaGndiO0Sks/S4tDjhNstYH7oGjWxi63cKqtGPY/p
WSTLpwrd4SY=
=vkT4
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] stdlib socket usage and "keepalive"

[Guido van Rossum]

Are you sure about this? ISTM that in most cases when a server goes
away unexpectedly the local host will discover this when it next tries
to use the socket. Also I recall reading that keepalives are a very
controversial concept (since they may actually break connections
unnecessarily if the internet merely has a hiccup).

--Guido

On Mon, Apr 12, 2010 at 2:51 PM, Jesus Cea  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Debugging a strange problem today, I got the following result:
>
> Sockets open by stdlib libraries are open without the "keepalive"
> option, so the system default is used. The system default under linux is
> "no keepalive".
>
> So, if you are using a URLlib connection, POP3 connection, IMAP
> connection, etc., any stdlib that internally creates a socket, and your
> server goes away suddendly (you lose network connectivity, by instance),
> the library will wait FOREVER for the server. The client can't detect
> that the server is not longer available.
>
> The "keepalive" option will send a probe packed every X minutes of
> inactivity, to check if the other side is still alive, even if the
> connection is idle.
>
> The issue is bad, but the solution is simple enough. Options:
>
> 1. All "client" libraries should create sockets with the "KEEPALIVE" option.
>
> 2. Modify the socket C module to create all sockets as "Keepalive" by
> default.
>
> 3. To have a global variable in the socket module to change the default
> for future sockets. Something like current "socket.setdefaulttimeout()".
> The default should be "keepalive".
>
> 4. Modify client libraries to accept a new optional socket-like object
> as an optional parameter. This would allow things like transparent
> compression or encryption, or to replace the socket connection by
> anything else (read/write to shared memory or database, for example).
>
> This is an issue in Linux because by default the sockets are not
> "keepalive". In other Unix systems, the default is "keepalive". I don't
> know about MS Windows.
>
> What do you think?. The solution seems trivial, after deciding the right
> way to go.
>
> PS: "socket.setdefaulttimeout()" is not enough, because it could
> shutdown a perfectly functional connection, just because it was idle for
> too long.
>
> - --
> Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
> j...@jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
> jabber / xmpp:j...@jabber.org         _/_/    _/_/          _/_/_/_/_/
> .                              _/_/  _/_/    _/_/          _/_/  _/_/
> "Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
> "My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQCVAwUBS8OV2Zlgi5GaxT1NAQIJhwP8CH+gij4KfrJ1oHW5ys4PboH5Ru0pgdly
> Wbsza0+uj3p68P1vDnC9jIr7j+fI3ql3DOc8zUoIKGpJoaWVspbv3c4vI4ATLHo+
> J6I18dpkviRT8/sT/69vMvghaGndiO0Sks/S4tDjhNstYH7oGjWxi63cKqtGPY/p
> WSTLpwrd4SY=
> =vkT4
> -END PGP SIGNATURE-
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] stdlib socket usage and "keepalive"

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/13/2010 12:09 AM, Guido van Rossum wrote:
> Are you sure about this? ISTM that in most cases when a server goes
> away unexpectedly the local host will discover this when it next tries
> to use the socket. Also I recall reading that keepalives are a very
> controversial concept (since they may actually break connections
> unnecessarily if the internet merely has a hiccup).

The case is this:

1. The client does a request. Wait "read"ing the answer.

2. The request is "slow", so the server "acks" the TCP datagram and
start to process the request.

3. Now the connection is idle. The server is working and the client is
waiting, blocked in the socket "read()".

4. Now you switch off the server (or unplug the ethernet wire).

5. If the client kernel is not doing "keepalive", the client will be
blocked FOREVER (days), waiting for the reply. If the client uses
"keepalive", its kernel will send a probe after a while (30 minutes, for
instance), the probe will fail, and the kernel will shutdown the
connection. The problem is: linux doesn't uses KEEPALIVE by default.

I have validated this behaviour with Ubuntu 9.10 and a network sniffer.

About controversial... keepalive are usually sent only when the
connection is 100% idle for a while, when "while" can be >15 minutes, so
the load should be "none" for regular connections.

- -- 
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBS8Of85lgi5GaxT1NAQJaKwP+P9WvbWqmRyHRvNJFB2wLj87EanyOIN1e
TP646wUHJQtuU3XCyAWF53uM4rDGsbVh9j4TGK7+C1SHoRpPHlLMUdfwddtk8nK3
Owmo0V10sfHrFi1E0D5Ub/LXd1GG0ec7Q7OGN30nUR//hCuLe57ckEodwNQGzmtA
As5yJ5iwGFw=
=GOP1
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] stdlib socket usage and "keepalive"

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/13/2010 12:09 AM, Guido van Rossum wrote:
> Also I recall reading that keepalives are a very
> controversial concept (since they may actually break connections
> unnecessarily if the internet merely has a hiccup).

That is true, but parameters are usually very conservative.

In standard Ubuntu box I host out there, the (default) parameters used
(if I activate keepalive):

+ Send the keepalive only after 1800 seconds of complete connection
inactivity.

+ After starting sending keepalives, send NINE probes.

+ Between probes, wait at least 75 seconds.

So you have to have 30 minutes of idle and then at least 11 minutes of
no conectivity to "lose" the connection.

- -- 
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBS8OhpJlgi5GaxT1NAQIb9AP7BLUAFzsltPr1omW+4+Ox7gF/lLDqNR5V
DXejCPJ2oBZdyebcwCSS1djh0thks8nRzG7oq61eA4c+Ax5mvsvW0cY5+BfNcrds
j6HwVJK+zgTX6NiO7VdGEysxfYHLbJcJ7PfoOHRWhYzolA7VSJriy8kfvgretTZQ
qJreRMaPai8=
=MQPh
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Fixing" the new GIL

[Nir Aides]

There is no need for the "forced" switch to be guaranteed. The motivation
for the wait is to increase throughput and decrease latency when OS
schedules next thread to run on same core as yielding thread but yielding
thread is about to continue running into CPU intensive library code. An
occasional spurious wakeup resulting in missed switch will not affect
that. Note
similar logic in new GIL.

I will add the loop to keep the code clean.

Thanks,
Nir



On Mon, Apr 12, 2010 at 10:49 PM, Peter Portante  wrote:

>  That code will not switch to another thread if the condition variable
> either does not actually block the thread on the call (which is allowed by
> the standard to give implementations some flexibility for making it work
> correctly – read the standard reasoning for more information), or the thread
> is woken up without a predicate change (also allowed by the standard for
> similar reasons). Both of those cases are called “spurious wake-ups”.
>
> You may not be able to readily get your implementation to behavior that
> way, but in the wild, we need to account for this behavior because Cpython
> will be run on systems where it will happen. :)
>
> -peter
>
>
>
> On 4/12/10 3:36 PM, "Nir Aides"  wrote:
>
> Please describe clearly a step by step scenario in which that code will
> fail.
>
>
> On Mon, Apr 12, 2010 at 10:25 PM, Peter Portante <
> peter.a.porta...@gmail.com> wrote:
>
> Yes, but unless you loop until the predicate is False, no forced switch is
> guaranteed to occur. You might as well remove the code. If you want to keep
> the code as is, call me when you need a life guard to help debug mystifying
> behaviors. ;) -peter
>
>
>
> On 4/12/10 3:17 PM, "Nir Aides" http://n...@winpdb.org> >
> wrote:
>
> The loop-less wait is similar to the one in new GIL. It is used to force a
> switch to next thread in particular scenario and the motivation is explained
> in comment to another if clause a few lines up. Those two if clauses can be
> joined though.
>
>
> On Mon, Apr 12, 2010 at 3:37 PM, Peter Portante <
> peter.a.porta...@gmail.com  > wrote:
>
> Hmm, so I see in
> bfs_yield():
>
> +if (tstate != NULL && bfs_thread_switch == tstate) {
> +COND_RESET(tstate->bfs_cond);
> +COND_WAIT(tstate->bfs_cond, bfs_mutex);
> +}
>
> So, to me, the above code says, “Wait for the condition that tstate is
> either NULL, or bfs_thread_switch does not equal tstate”. So the predicate
> is: “(tstate != NULL && bfs_thread_switch == tstate)”.
>
> If the predicate is True before you call COND_WAIT() and True after you
> call COND_WAIT(), either you don’t need to call COND_WAIT() at all, or you
> need to loop until the predicate is False. There is no guarantee that a
> condition wait actually did anything at all. Yes, there will be spurious
> wake ups, but you can’t tell if the thread actually blocked and then woke
> up, or never blocked at all. If it never actually blocks, then that code
> path is not helpful.
>
> On Windows, before this loop in bfs_schedule():
>
> +COND_RESET(tstate->bfs_cond);
> +while (bfs_thread != tstate) {
> +_bfs_timed_wait(tstate, timestamp);
> +timestamp = get_timestamp();
> +}
>
> You might want to avoid the call to reset the condition variable if the
> predicate is already False.
>
> -peter
>
>
>
> On 4/12/10 8:12 AM, "Nir Aides" http://n...@winpdb.org>  <
> http://n...@winpdb.org> > wrote:
>
> Hi Peter,
>
> There is no need for a loop in bfs_yield().
>
>
> On Mon, Apr 12, 2010 at 4:26 AM, Peter Portante <
> peter.a.porta...@gmail.com   <
> http://peter.a.porta...@gmail.com> > wrote:
>
> Nir,
>
> Per the POSIX standard, both pthread_cond_wait() and
> pthread_cond_timedwait() need to be performed in a loop.  See the fourth
> paragraph of the description from:
>
>
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html<
> http://www.opengroup.org/onlinepubs/95399/functions/pthread_cond_timedwait.html>
>
>
>
> For the Windows side, I think you have a similar problem. Condition
> variables are signaling mechanisms, and so they have a separate boolean
> predicate associated with them. If you release the mutex that protects the
> predicate, then after you reacquire the mutex, you have to reevaluate the
> predicate to ensure that the condition has actually been met.
>
> You might want to look at the following for a discussion (not sure how good
> it is, as I just google’d it quickly) of how to implement POSIX semantics on
> Windows:
>
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html <
> http://www.cs.wustl.edu/~schmidt/win32-cv-1.html>
>
>
> Before you can evaluate the effectiveness of any of the proposed scheduling
> schemes, the fundamental uses of mutexes and condition variables, and their
> implementations, must be sound.
>
> -peter
>
>
>
> On 4/11/10 6:50 PM, "Nir Aides" <
>
> n...@winpdb.org   

Re: [Python-Dev] stdlib socket usage and "keepalive"

[Antoine Pitrou]

Jesus Cea  jcea.es> writes:
> 
> PS: "socket.setdefaulttimeout()" is not enough, because it could
> shutdown a perfectly functional connection, just because it was idle for
> too long.

The socket timeout doesn't shutdown anything. It just puts a limit on how much
time recv() and send() can block. Then it's up to you to detect whether the
server is still alive (for example by pinging it through whatever means the
application protocol gives you).

> 1. All "client" libraries should create sockets with the "KEEPALIVE" option.
> 
> 2. Modify the socket C module to create all sockets as "Keepalive" by
> default.

I don't know whether there are any negative implications of such solutions.
Granted, the first one is less radical than the second (because servers wouldn't
be impacted).

> 4. Modify client libraries to accept a new optional socket-like object
> as an optional parameter. This would allow things like transparent
> compression or encryption, or to replace the socket connection by
> anything else (read/write to shared memory or database, for example).

This could be useful too.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] stdlib socket usage and "keepalive"

[Daniel Stutzbach]

On Mon, Apr 12, 2010 at 5:34 PM, Jesus Cea  wrote:

> The problem is: linux doesn't uses KEEPALIVE by default.
>

If you believe the problem is with the Linux kernel, perhaps you should take
up your case on a more appropriate mailing list?

Python's socket module is a fairly low-level module, as it's just a thin
wrapper around the corresponding operating system calls.  Anyone using it
has to be prepared to deal with a certain amount of exposed operating system
details.

If you want to use TCP KEEPALIVE on Linux, then just call:
my_socket_object.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)

Most non-trivial applications use select() or poll() to avoid blocking calls
and do their own timeout-checking at the application layer, so they don't
need KEEPALIVE.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] stdlib socket usage and "keepalive"

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/13/2010 12:59 AM, Daniel Stutzbach wrote:
> Most non-trivial applications use select() or poll() to avoid blocking
> calls and do their own timeout-checking at the application layer, so
> they don't need KEEPALIVE.

I am thinking about python stdlibs like imaplib, poplib, smtplib,
ftplib, urllib, etc., etc., ...

- -- 
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBS8On2Zlgi5GaxT1NAQJDhQQAkveJ2pF9wZGSmQTCW1HFpPX4MOtaEjxs
AFCLX3WcFf2qwpN82hLluJRfkmIWZDHhWU4bKJ/GKRkGvsWb3zcOHaX0solpibqK
yS/gVUsrgd1GuqxyQqhtd4J5+fPwZr5RQ5JrO/PjpLH8CgTq6azjufixO4Cve4Jh
X4LO3GMekj8=
=ws3T
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Getting an optional parameter instead of creating a socket internally (was: Re: stdlib socket usage and " keepalive" )

[Jesus Cea]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/13/2010 12:47 AM, Antoine Pitrou wrote:
> Jesus Cea  jcea.es> writes:
>>
>> PS: "socket.setdefaulttimeout()" is not enough, because it could
>> shutdown a perfectly functional connection, just because it was idle for
>> too long.
> 
> The socket timeout doesn't shutdown anything. It just puts a limit on how much
> time recv() and send() can block. Then it's up to you to detect whether the
> server is still alive (for example by pinging it through whatever means the
> application protocol gives you).

A regular standard library (let say, poplib) would abort, after getting
the timeout exception.

>> 4. Modify client libraries to accept a new optional socket-like object
>> as an optional parameter. This would allow things like transparent
>> compression or encryption, or to replace the socket connection by
>> anything else (read/write to shared memory or database, for example).
> 
> This could be useful too.

I have been thinking about this for years. Do you actually think this
could be formally proposed?.

What bugs me is that the socket creation is deep inside the stdlibs. You
can not control it easily (I have overloaded socket.socket() in the past
for controlling the number of concurrent connections to servers, for
instance, or providing encryption), and it is difficult to test.

If these stdlib methods could accept an optional parameter instead of
creating the socket internally, test is trivial, and you can reuse the
lib to access the service via an arbitrary object (this weekend I just
tunneled TCP/IP via DNS requests/answers, shame on you, airport wifi
hotspots!).

- -- 
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBS8Oqi5lgi5GaxT1NAQKPpgP/S2V3umt4SpOw+Epxtj3/oxEFcQuh3s1z
WNvS59+qY6IK0+/MCvxDiAYliGbj8PY76lXmRodJOzwVFe7uPzLhq4h3gBBv0zXs
KvBHdyL5fnp4UEId89L7+SqejgAfpJk3GXYwAnpLyF5iQzaiYzp0rNDOuSBCeAmW
jFjzjMMZznQ=
=cFb/
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] stdlib socket usage and "keepalive"

[Guido van Rossum]

On Mon, Apr 12, 2010 at 3:59 PM, Daniel Stutzbach
 wrote:
> On Mon, Apr 12, 2010 at 5:34 PM, Jesus Cea  wrote:
>>
>> The problem is: linux doesn't uses KEEPALIVE by default.
>
> If you believe the problem is with the Linux kernel, perhaps you should take
> up your case on a more appropriate mailing list?
>
> Python's socket module is a fairly low-level module, as it's just a thin
> wrapper around the corresponding operating system calls.  Anyone using it
> has to be prepared to deal with a certain amount of exposed operating system
> details.

Bingo. I expect that changing this will have too many unanticipated
ramifications to be safe.

> If you want to use TCP KEEPALIVE on Linux, then just call:
>     my_socket_object.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
>
> Most non-trivial applications use select() or poll() to avoid blocking calls
> and do their own timeout-checking at the application layer, so they don't
> need KEEPALIVE.

> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting an optional parameter instead of creating a socket internally (was: Re: stdlib socket usage and " keepalive" )

[Guido van Rossum]

On Mon, Apr 12, 2010 at 4:19 PM, Jesus Cea  wrote:
> On 04/13/2010 12:47 AM, Antoine Pitrou wrote:
>> Jesus Cea  jcea.es> writes:
>>>
>>> PS: "socket.setdefaulttimeout()" is not enough, because it could
>>> shutdown a perfectly functional connection, just because it was idle for
>>> too long.
>>
>> The socket timeout doesn't shutdown anything. It just puts a limit on how 
>> much
>> time recv() and send() can block. Then it's up to you to detect whether the
>> server is still alive (for example by pinging it through whatever means the
>> application protocol gives you).
>
> A regular standard library (let say, poplib) would abort, after getting
> the timeout exception.
>
>>> 4. Modify client libraries to accept a new optional socket-like object
>>> as an optional parameter. This would allow things like transparent
>>> compression or encryption, or to replace the socket connection by
>>> anything else (read/write to shared memory or database, for example).
>>
>> This could be useful too.
>
> I have been thinking about this for years. Do you actually think this
> could be formally proposed?.
>
> What bugs me is that the socket creation is deep inside the stdlibs. You
> can not control it easily (I have overloaded socket.socket() in the past
> for controlling the number of concurrent connections to servers, for
> instance, or providing encryption), and it is difficult to test.
>
> If these stdlib methods could accept an optional parameter instead of
> creating the socket internally, test is trivial, and you can reuse the
> lib to access the service via an arbitrary object (this weekend I just
> tunneled TCP/IP via DNS requests/answers, shame on you, airport wifi
> hotspots!).

Yeah, this sounds like a useful change. Would be very useful for testing too.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting an optional parameter instead of creating a socket internally (was: Re: stdlib socket usage and " keepalive" )

[exarkun]

On 12 Apr, 11:19 pm, j...@jcea.es wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/13/2010 12:47 AM, Antoine Pitrou wrote:

Jesus Cea  jcea.es> writes:


PS: "socket.setdefaulttimeout()" is not enough, because it could
shutdown a perfectly functional connection, just because it was idle 
for

too long.


The socket timeout doesn't shutdown anything. It just puts a limit on 
how much
time recv() and send() can block. Then it's up to you to detect 
whether the
server is still alive (for example by pinging it through whatever 
means the

application protocol gives you).


A regular standard library (let say, poplib) would abort, after getting
the timeout exception.
4. Modify client libraries to accept a new optional socket-like 
object

as an optional parameter. This would allow things like transparent
compression or encryption, or to replace the socket connection by
anything else (read/write to shared memory or database, for example).


This could be useful too.


I have been thinking about this for years. Do you actually think this
could be formally proposed?.


Every once in a while I make a little bit more progress on the PEP I'm 
working on for this.  If you want to talk more about this, you can find 
me in #python-dev or #twisted on freenode.


Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com