[Python-Dev] python running in several threads

[Martin Aliger]

Hello Python developers,

I know this is developers list and I hope I'm right here. I'm using
python as scripting engine embedded in application but issue should be
more general.

When several scripts are executed simultaneously (under several
threads) one have to acquire/release locks. This degrades performance
IMHO, mainly on server installations (many web servers use python
engine). Moreover those scripts are commonly independent chunks of
code which do not create other threats. Under those limitations, do
you think, an easy rewrite of execution engine is possible to allow
simultaneous run of all threads?

I checked the sources esp. PyThreadState object and it seems all those
variables could be held locally. Another way is using thread local
storage, but it could be a little problematic with porting issues.

How do you see it? Do you have any plans here?

Thanks and regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python running in several threads

[Martin Aliger]

Thanks both for answers,

I read some articles you link and I'd to say - alot discussion around!

I also found some notes that mod_python use some technique other than
GIL to allow multi-threading in Apache 2.0. Does someone know anything
closer? Any links or so?

Sorry I raise this old stuff again.

Martin


Just short notes:
> Removing the GIL is easy, but then the interpreter crashes in cases of
> simultaneous accesses to dictionaries, reference counters, etc.

I know. But under mine conditions is impossible to share dictionaty or
ref.counters so this do not happen. This dont happen neither on web
server running simmultaniously several .py scripts (same conditions)
 
> This is just not true. On a single-processor machine, the GIL does
> *not* degrade performance. Instead, it increases throughput (and
> thus performance).

Under bigger load yes. It degrades a little under small load, but its
usually not big problem.
 
> On a multi-processor machine, you often use multiple operating
> system processes to serve request (e.g. in CGI or Apache mpm-prefork,
> or even the typical mpm-worker configuration). If you then have
> different processes running Python, they don't interfere with
> each other at all.

Unfortunatelly many configurations dont do that. I run several servers
with hyperthreading processors and none of them runs two IISs or
apaches just for load-balance python scripts.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com