urllib (in thread) never returns
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
I've been having an intermittent problem with urllib.
With an interval of around 15 minutes (i.e.: this is run every 15m)
this code runs fine for about 1-2 weeks, but then gets it's
knickers in a twist, and never seems to return, nor except.
try:
log("ABOUT TO USE urllib TO FETCH ["+self.url+"]")
f = urllib.urlopen(self.url)
temp_xml = f.read()
[1] log("DONE! GOT %d bytes for [%s]" % (len(str(temp_xml)),self.url))
f.close()
refreshed = True
except:
[2] log("Error: fetching ["+self.url+"], retrying ... ")
...
In the 'broken' state, I never see the log message [1] or [2], it
just sits in either the urlopen() or read() forever.
The code is running in a separate thread.
TCP timeout is set to 45 seconds.
Running on Linux (gentoo)
What I think is wrong, is that the server is sitting
behind a somewhat-dodgey ADSL connection. The server it's
contacting is also on a dodgey ADSL connection.
I'm guessing that the socket is opened with
the ADSL modem is NATing the connection, but when
the backend internet connection bounces, the NAT'ed
connection somehow stays up, leaving the connection
somehow dangling; alive, but dead too...
I would have thought that some urllib-internal timeout would
fix this?!
I could watch the thread from another thread, implementing
my own timeout... But then (AFAIK) there's no way to terminate
a mis-behaving thread anyways.
Any suggestions?
thanks,
- -Kingsley
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFEut+ISo2jKyi4JlgRApIdAJ4iIWDLXR9am659XAS1ajLv1ry12wCfeOiI
FUNbrisdyo4j3Yle4zN2ESk=
=EPU2
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to convert a list of strings into a list of variables
noydb wrote: > How would you convert a list of strings into a list of variables using > the same name of the strings? > > So, ["red", "one", "maple"] into [red, one, maple] > > Thanks for any help! red="a string" one="another string" maple="a file path" old=["red", "one", "maple"] newList=map(eval, old) -- http://mail.python.org/mailman/listinfo/python-list
Re: Help needed - To get path of a directory
On 13/10/10 15:26, Bishwarup Banerjee wrote:
I want to get the absolute path of the Directory I pass explicitly. Like
functionName("\abcd").
I should pass the name of the directory and the function should search for
it in the Hard drives and return me the full path of location on the
drive. I tried using os.path, but didn't succeed.
One way to achieve this is to fetch a recursive directory list for all
drives, and then search for your directory_name.endswith("abcd") in each
list. But what would you do with multiple occurrences ?
Here's a function to fetch a recursive directory list:
import os.path
def getRecursiveDirList(path):
dir_list = []
if (os.path.isdir(path)):
try:
files = os.listdir(path)
files.sort()
except:
files = []
for x in files:
full_path = os.path.join(path, x)
if (os.path.isdir(full_path)):
dir_list.append(os.path.join(full_path,''))
for full_path in getRecursiveDirList(full_path):
dir_list.append(full_path)
return dir_list
I've only tested this function under Linux.
It's also probably sub-optimal.
I don't know how to enumerate all your windows device letters.
regards,
-kt
--
http://mail.python.org/mailman/listinfo/python-list
GCC process not working as expected when called in Python (3.1.2) subprocess-shell, but OK otherwise
Hi,
I'm using GCC as a pre-processor for a C-like language (EDDL) to handle all
the includes, macros, etc. producing a single source file for another
compiler. My python code massages the inputs (which arrive in a .zip file),
then calls GCC.
I have a problem where if I call GCC from my python script some of the
#defines are not processed in the output. However if I paste the exact same
GCC command-line into a shell, I get a correct output.
I'm calling GCC in this manner:
### Execute GCC, keep stdout & stderr
err_out = open(error_filename,"wb")
process = subprocess.Popen(gcc_command, stderr=err_out, bufsize=81920,
cwd=global_options['tmp'])
gcc_exit_code = process.wait()
log("GCC Exit Code %u" % (gcc_exit_code))
err_out.close()
where gcc_command is:
"/usr/bin/gcc -I /tmp/dd-compile_1286930109.99 -I
/home/foo/eddl-includes -D__TOKVER__=600 -ansi -nostdinc -v -x c -E -o
/tmp/dd-compile_1286930109.99/11130201.ddl.OUT
/tmp/dd-compile_1286930109.99/11130201.ddl"
So when this code spawns GCC, the compiler does not really work 100%, but if
I paste this exact command line, the output is perfect.
I'm not really sure how to debug this. I already checked the ulimits, and
permissions shouldn't be a problem since it's all run by the same user, I
also checked the environment - these were copied into the subshell.
GCC produces no warnings, or errors. The output is mostly OK, some other
macros have been processed.
If I diff the working output with the non-working one, the differences are
only a bunch of skipped #defines.
gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
Python 3.1.2 (release31-maint, Sep 17 2010, 20:27:33)
Linux 2.6.35-22-generic #34-Ubuntu SMP Sun Oct 10 09:26:05 UTC 2010 x86_64
GNU/Linux
Any suggestions for helping me debug this would be much appreciated.
thanks,
-kt
PS> this query has also been posted to the GCC-help list
--
http://mail.python.org/mailman/listinfo/python-list
