KeyboardInterrupt catch does not shut down the socketserver
I have problems in getting a SocketServer to shutdown. Why does this not
actually stop the application?
from SocketServer import UnixStreamServer, BaseRequestHandler
server = UnixStreamServer('/tmp/ss.sock', BaseRequestHandler)
try:
server.serve_forever()
except KeyboardInterrupt:
server.shutdown()
After that the server does not respond any more, but the application hangs.
What's the proper way to shutdown the socketserver and what is my mistake?
--
http://mail.python.org/mailman/listinfo/python-list
Re: KeyboardInterrupt catch does not shut down the socketserver
Lawrence D'Oliveiro wrote: In message , Igor Katson wrote: I have problems in getting a SocketServer to shutdown. Do you want to do a shutdown or a close? I want the server close the socket, and the program to continue after that (in this case, just to terminate). -- http://mail.python.org/mailman/listinfo/python-list
Re: KeyboardInterrupt catch does not shut down the socketserver
Lawrence D'Oliveiro wrote: In message , Igor Katson wrote: Lawrence D'Oliveiro wrote: In message , Igor Katson wrote: I have problems in getting a SocketServer to shutdown. Do you want to do a shutdown or a close? I want the server close the socket ... You want to do a close, do a close, not a shutdown <http://docs.python.org/library/socket.html>. Shutdown implies closing the listening socket, doesn't it? -- http://mail.python.org/mailman/listinfo/python-list
Re: KeyboardInterrupt catch does not shut down the socketserver
Gabriel Genellina wrote:
En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribió:
Lawrence D'Oliveiro wrote:
In message ,
Igor Katson wrote:
Lawrence D'Oliveiro wrote:
In message ,
Igor Katson wrote:
I have problems in getting a SocketServer to shutdown.
Do you want to do a shutdown or a close?
I want the server close the socket ...
You want to do a close, do a close, not a shutdown
<http://docs.python.org/library/socket.html>.
Shutdown implies closing the listening socket, doesn't it?
No (perhaps it should, but that is another issue). There is a
documentation bug; BaseServer.shutdown is documented as "Tells the
serve_forever() loop to stop and waits until it does." [1]
The docstring is much more explicit: """Stops the serve_forever loop.
Blocks until the loop has finished. This must be called while
serve_forever() is running in another thread, or it will deadlock."""
So, if you have a single-threaded server, *don't* use shutdown(). And, to
orderly close the listening socket, use server_close() instead. Your code
would become:
from SocketServer import TCPServer, BaseRequestHandler
server = TCPServer(('localhost',1234), BaseRequestHandler)
try:
server.serve_forever()
except KeyboardInterrupt:
print "^C detected"
pass
finally:
print "server_close()"
server.server_close()
print "bye"
(I've opened http://bugs.python.org/issue6031 )
[1]
http://docs.python.org/dev/library/socketserver.html?highlight=baseserver#SocketServer.BaseServer.shutdown
Thanks, Gabriel, that's exactly what I was waiting for.
--
http://mail.python.org/mailman/listinfo/python-list
Re: KeyboardInterrupt catch does not shut down the socketserver
Gabriel Genellina wrote:
En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribió:
Lawrence D'Oliveiro wrote:
In message ,
Igor Katson wrote:
Lawrence D'Oliveiro wrote:
In message ,
Igor Katson wrote:
I have problems in getting a SocketServer to shutdown.
Do you want to do a shutdown or a close?
I want the server close the socket ...
You want to do a close, do a close, not a shutdown
<http://docs.python.org/library/socket.html>.
Shutdown implies closing the listening socket, doesn't it?
No (perhaps it should, but that is another issue). There is a
documentation bug; BaseServer.shutdown is documented as "Tells the
serve_forever() loop to stop and waits until it does." [1]
The docstring is much more explicit: """Stops the serve_forever loop.
Blocks until the loop has finished. This must be called while
serve_forever() is running in another thread, or it will deadlock."""
So, if you have a single-threaded server, *don't* use shutdown(). And, to
orderly close the listening socket, use server_close() instead. Your code
would become:
from SocketServer import TCPServer, BaseRequestHandler
server = TCPServer(('localhost',1234), BaseRequestHandler)
try:
server.serve_forever()
except KeyboardInterrupt:
print "^C detected"
pass
finally:
print "server_close()"
server.server_close()
print "bye"
(I've opened http://bugs.python.org/issue6031 )
[1]
http://docs.python.org/dev/library/socketserver.html?highlight=baseserver#SocketServer.BaseServer.shutdown
Hmm. Gabriel, could you please show the same for the threaded version?
This one deadlocks:
from SocketServer import TCPServer, BaseRequestHandler
from threading import Thread
server = TCPServer(('localhost',1234), BaseRequestHandler)
try:
run_thread = Thread(target=server.serve_forever)
run_thread.start()
run_thread.join()
except KeyboardInterrupt:
print "^C detected"
pass
finally:
print "server_shutdown()"
kill_thread = Thread(target=server.shutdown)
kill_thread.start()
print "bye"
--
http://mail.python.org/mailman/listinfo/python-list
Re: KeyboardInterrupt catch does not shut down the socketserver
Gabriel Genellina wrote: En Sat, 16 May 2009 04:04:03 -0300, Igor Katson escribió: Gabriel Genellina wrote: En Fri, 15 May 2009 09:04:05 -0300, Igor Katson escribió: Lawrence D'Oliveiro wrote: In message , Igor Katson wrote: Lawrence D'Oliveiro wrote: In message , Igor Katson wrote: I have problems in getting a SocketServer to shutdown. Shutdown implies closing the listening socket, doesn't it? No (perhaps it should, but that is another issue). There is a documentation bug; BaseServer.shutdown is documented as "Tells the serve_forever() loop to stop and waits until it does." [1] The docstring is much more explicit: """Stops the serve_forever loop. Blocks until the loop has finished. This must be called while serve_forever() is running in another thread, or it will deadlock.""" So, if you have a single-threaded server, *don't* use shutdown(). And, to orderly close the listening socket, use server_close() instead. Your Hmm. Gabriel, could you please show the same for the threaded version? This one deadlocks: [code removed] The shutdown method should *only* be called while serve_forever is running. If called after server_forever exited, shutdown() blocks forever. [code removed] But, what are you after, exactly? I think I'd use the above code only in a GUI application with a background server. There are other alternatives, like asyncore or Twisted. For now, I am just using server.server_close() and it works. The server itself is an external transaction manager for PostgreSQL, when a client connects to it, serialized data interchange beetween the server and the client starts, e.g. first the client sends data, then the server sends data, then again the client, then the server and so on. I haven't used asyncore or Twisted yet, and didn't know about their possible usage while writing the project. I'll research in that direction. -- http://mail.python.org/mailman/listinfo/python-list
How to reuse TCP listening socket immediately after it was connected at least once?
I have written a socket server and some arbitrary clients. When I shutdown the server, and do socket.close(), I cannot immediately start it again cause it has some open sockets in TIME_WAIT state. It throws address already in use exception at me. I have searched for that in google but haven't found a way to solve that. Tried setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) but that does not help. Is there a nice way to overcome this? -- http://mail.python.org/mailman/listinfo/python-list
Re: How to reuse TCP listening socket immediately after it was connected at least once?
Igor Katson wrote: I have written a socket server and some arbitrary clients. When I shutdown the server, and do socket.close(), I cannot immediately start it again cause it has some open sockets in TIME_WAIT state. It throws address already in use exception at me. I have searched for that in google but haven't found a way to solve that. Tried setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) but that does not help. Is there a nice way to overcome this? Solved myself. SO_REUSEADDE should be used on the second listening socket creation (while time_wait already hangs) -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing and file I/O
Infinity77 wrote:
Hi All,
I am trying to speed up some code which reads a bunch of data from
a disk file. Just for the fun of it, I thought to try and use parallel
I/O to split the reading of the file between multiple processes.
Although I have been warned that concurrent access by multiple
processes to the same file may actually slow down the reading of the
file, I was curious to try some timings by varying the number of
processes which read the file. I know almost nothing of
multiprocessing, so I was wondering if anyone had some very simple
snippet of code which demonstrates how to read a file using
multiprocessing.
My idea was to create a "big" file by doing:
fid = open("somefile.txt", "wb")
fid.write("HELLO\n"*1e7)
fid.close()
and then using fid.seek() to point every process I start to a position
inside the file and start reading from there. For example, with 4
processes and a 10 MB file, I would tell the first process to read
from byte 0 to byte 2.5 million, the second one from 2.5 million to 5
million and so on. I just have an academic curiosity :-D
Any suggestion is very welcome, either to the approach or to the
actual implementation. Thank you for your help.
Andrea.
If the thing you would want to speed up is the processing of the file
(and not the IO), I would make one process actually read the file, and
feed the other processes with the data from the file through a queue.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to reuse TCP listening socket immediately after it was connected at least once?
Roy Smith wrote: In article , Lawrence D'Oliveiro wrote: In message , Igor Katson wrote: I have written a socket server and some arbitrary clients. When I shutdown the server, and do socket.close(), I cannot immediately start it again cause it has some open sockets in TIME_WAIT state. It throws address already in use exception at me. There's a reason for that. It's to ensure that there are no leftover packets floating around the Internet somewhere, that you might mistakenly receive and think they were part of a new connection, when they were in fact part of an old one. In theory, that is indeed the reason for the TIME_WAIT state. In practice, however, using SO_REUSEADDR is pretty safe, and common practice. You've got several things working in your favor. First, late-delivery of packets is pretty rare. Second, if some late packet were to arrive, the chances of them having the same local and remote port numbers as an existing connection is slim. And, finally, the TCP sequence number won't line up. One thing to be aware of is that SO_REUSEADDR isn't 100% portable. There are some systems (ISTR HP-UX) which use SO_REUSEPORT instead of SO_REUSEADDR. The original specifications weren't very clear, and some implementers read them in strange ways. Some of that old code continues in use today. I only mention this because if you try SO_REUSEADDR and it's not doing what you expect, it's worth trying SO_REUSEPORT (or both) to see what happens on your particular system. The right thing to do is try to ensure that all your connections are properly closed at shutdown. That may not be enough (if your server crashes due to bugs), so the other thing you need to do is retry the socket open, say, at 30-second intervals, until it succeeds. That may be a reasonable thing to do for production code, but when you're building and debugging a server, it's a real pain to not be able to restart it quickly whenever you want (or need) to. Thanks for a great answer, Roy! -- http://mail.python.org/mailman/listinfo/python-list
Re: A fast way to read last line of gzip archive ?
So in summary, the choices when tested on my system ended up at: last 26 last-chunk2.7 last-chunk-2 2.3 last-popen1.7 last-gzip 1.48 last-decompress 1.12 So by being willing to mix in some more direct code with the GzipFile object, I was able to beat the overhead of shelling out to the faster utilities, while remaining in pure Python. -- David Though I didn't need this practically, it was a fantastic reading. Gorgeus, David! Thanks for the ultimative post. -- http://mail.python.org/mailman/listinfo/python-list
What is the purpose of "struct" and "array" modules
I pretty much understand what they do, but what's the case of using these modules by example? Is it something like pickle, to store the data efficiently in files? -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: pyTenjin 0.8.0 - much faster template engine than Django
kwatch wrote: I have released pyTenjin 0.8.0 Thanks for your project. I have used it a little, and there is a question to you. import tenjin from tenjin.helpers import * shared_cache = tenjin.GaeMemcacheCacheStorage() engine = tenjin.Engine(cache=shared_cache) 1. Why should I import tenjin.helpers if I don't use the helpers in my code? 2. Why does the code not work if I don't import the helpers? I think you should manage this issue inside the library. -- http://mail.python.org/mailman/listinfo/python-list
