hi i am using urllib2 to do some automated web thing. basically i hit on sites and check the price what they are offering for their product and then decide if i want to lower or increase my pricing.so in short i have to hit hundreds of sites!!!!!
for the problem: ============= i run 20 threads all do the same stuff. (hit and run :) ) after around 10-15 hits(per thread) hits the thread does nothing. it freezes. slowely but STEADILY all the threads end up with the same fate :( i did some netstat and found out that the connecton(sockets) the program had opened are waiting the CLOSE_WAIT state !! netstat -t tcp 1 0 192.168.1.2:4882 host-blabla:www CLOSE_WAIT tcp 1 0 192.168.1.2:4884 host-blabla:www CLOSE_WAIT tcp 1 0 192.168.1.2:4375 host-blabla:www CLOSE_WAIT OUTPUT OF PROGRAM: THREAD: #Thread-2 getting price from webi-d 7511975 DONE !!! THREAD: #Thread-1 getting price from webi-d 4449152 DONE !!! THREAD: #Thread-2 getting price from webi-d 7466091 DONE !!! THREAD: #Thread-1 getting price from webi-d 8641914 DONE !!! THREAD: #Thread-2 getting price from webi-d 7745289 DONE !!! THREAD: #Thread-1 getting price from webi-d 6032442 DONE !!! THREAD: #Thread-2 getting price from webi-d 8149873 DONE !!! no-price-on-page error THREAD: #Thread-1 getting price from webi-d 5842934 DONE !!! no-price-on-page error THREAD: #Thread-2 getting price from webi-d 3385778 DONE !!! THREAD: #Thread-1 getting price from webi-d 4610122 DONE !!! THREAD: #Thread-2 getting price from webi-d 8641536 DONE !!! THREAD: #Thread-1 getting price from webi-d 4219935 DONE !!! ---------and thats it, it freezes. i have waited 1hr the sockets have not changed their states! :( please help :) -- http://mail.python.org/mailman/listinfo/python-list
