On Tue, 17 Mar 2009 18:03:59 +0100, Laszlo Nagy <gand...@shopzeus.com> wrote:
For whatever reason, you're ending up with a lot of open files and/or
sockets
(and/or any other resource based on file descriptors). That results in new
file descriptors having large values (>=1024).
You cannot use select() with such file descriptors. Try poll() instead,
or Twisted. ;)
Poll is not good, because it does not wait.
This is not true. You can accomplish just what you're doing with select()
using poll() instead. Oops, but I forgot, Windows doesn't have poll(), so
that doesn't really help you. There's WaitForSingleObject if you can depend
on pywin32, though. It supports timeouts, just as select() does (in fact,
select() is a wrapper around WaitForMultipleObjects on Windows - the multi-
handle version of WaitForSingleObject).
[snip]
You are talking about twisted - can someone tell me how twisted does this?
Twisted does it by taking the call to low-level I/O routines out of your
hands and just doing them right on its own. ;) It also doesn't use blocking
I/O calls, and it doesn't make any I/O calls unless it thinks they're going
to succeed immediately anyway.
It deals with the problem of allowing timely exits on Windows by waiting
up select frequently, which is just a hack to work around the bug I
mentioned in CPython on Windows.
[snip]
I would really like to know what other options I have to implement this
efficiently. Is my approach bad?
I prefer the approach Twisted (and other libraries too) take. Don't use
threads to multiplex I/O. Use non-blocking I/O calls and select() or the
best platform-specific equivalent.
Or am I using the wrong language? I started to use threads heavily in the
past months. Haskell might be a better choice for me, except that Python is
much more elegant. :-)
I'm sure people will tell you that this is a good use of threads and that
Python is up to the task. I think it's a bad use of threads, but with a
different approach, I still think Python is up to the task.
Jean-Paul
--
http://mail.python.org/mailman/listinfo/python-list
