kj wrote:
I just spent about 1-1/2 hours tracking down a bug.
An innocuous little script, let's call it buggy.py, only 10 lines
long, and whose output should have been, at most two lines, was
quickly dumping tens of megabytes of non-printable characters to
my screen (aka gobbledygook), and in the process was messing up my
terminal *royally*. Here's buggy.py:
import sys
import psycopg2
connection_params = "dbname='%s' user='%s' password='%s'" % tuple(sys.argv[1:])
conn = psycopg2.connect(connection_params)
cur = conn.cursor()
cur.execute('SELECT * FROM version;')
print '\n'.join(x[-1] for x in cur.fetchall())
(Of course, buggy.py is pretty useless; I reduced the original,
more useful, script to this to help me debug it.)
Through a *lot* of trial an error I finally discovered that the
root cause of the problem was the fact that, in the same directory
as buggy.py, there is *another* innocuous little script, totally
unrelated, whose name happens to be numbers.py. (This second script
is one I wrote as part of a little Python tutorial I put together
months ago, and is not much more of a script than hello_world.py;
it's baby-steps for the absolute beginner. But apparently, it has
a killer name! I had completely forgotten about it.)
Both scripts live in a directory filled with *hundreds* little
one-off scripts like the two of them. I'll call this directory
myscripts in what follows.
It turns out that buggy.py imports psycopg2, as you can see, and
apparently psycopg2 (or something imported by psycopg2) tries to
import some standard Python module called numbers; instead it ends
up importing the innocent myscript/numbers.py, resulting in *absolute
mayhem*.
(This is no mere Python "wart"; this is a suppurating chancre, and
the fact that it remains unfixed is a neverending source of puzzlement
for me.)
How can the average Python programmer guard against this sort of
time-devouring bug in the future (while remaining a Python programmer)?
The only solution I can think of is to avoid like the plague the
basenames of all the 200 or so /usr/lib/pythonX.XX/xyz.py{,c} files,
and *pray* that whatever name one chooses for one's script does
not suddenly pop up in the appropriate /usr/lib/pythonX.XX directory
of a future release.
What else can one do? Let's see, one should put every script in its
own directory, thereby containing the damage.
Anything else?
Any suggestion would be appreciated.
TIA!
~k
Here's a pretty simple fix that should work in about any version of
python available:
Put modules in ~/lib. Put scripts in ~/bin. Your modules end with
.py. Your scripts don't. Your scripts add ~/lib to sys.path as
needed. Things that go in ~/lib are named carefully. Things in ~/bin
also need to be named carefully, but for an entirely different reason -
if you name something "ls", you may get into trouble.
Then things in ~/lib plainly could cause issues. Things in ~/bin don't.
Ending everything with .py seems to come from the perl tradition of
ending everything with .pl. This perl tradition appears to have come
from perl advocates wanting everyone to know (by looking at a URL) that
they are using a perl CGI. IMO, it's language vanity, and best
dispensed with - aside from this issue, it also keeps you from rewriting
your program in another language with an identical interface.
This does, however, appear to be a scary issue from a security
standpoint. I certainly hope that scripts running as root don't search
"." for modules.
--
http://mail.python.org/mailman/listinfo/python-list
