Greetings Ganesh,
>> You're falling into the trap of assuming that the only exception you
>> can ever get is the one that you're planning for, and then handling.
>
>Ok sure !
This point is very important, so I'll reiterate it. I hope the poor
horse lives.
>> ALL exceptions as though they were that one. Instead catch ONLY the
>> exception that you're expecting to see, and ignore everything else. Do
>> not use a bare "except:" clause, nor even "except Exception:", for
>> this. You will appreciate it later on.
>
>What option do I have now in so ensure that I loop over the For
>loop.
>Try except is ruled out ?
No, no not at all!
#1: Yes, you (probably) should be using the try-except construct.
#2: No, you should not (ever) use a bare try-except construct.
Please read below. I will take a stab at explaining the gaps of
understanding you seem to have (others have tried already, but I'll
try, as well).
I am going to give you four different functions which demonstrate
how to use exceptions. You may find it instructive to paste these
functions into an interactive Python shell and try them out, as
well.
I will explain a few different cases, which I will show below, but I
think you'll need to figure out how to apply this to your situation.
Step 1: catch a specific Exception
----------------------------------
def catchValueError(val, default=0):
'''somebody passed in a non-convertible type, return a default'''
try:
return int(val)
except ValueError:
return default
Now, let's try passing a few values into that function:
>>> catchValueError(0)
0
>>> catchValueError('-3')
-3
>>> catchValueError('Catullus')
0
The last example above is the most interesting. We fed a string to
the builtin function int() and it raised a ValueError. Our little
try-except block, however, caught the ValueError and performed our
desired action (in this case, returning a default value).
Step 2: catch a specific Exception
----------------------------------
But, what about this:
>>> catchValueError(dict())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in catchValueError
TypeError: int() argument must be a string or a number, not 'dict'
Now, you can see that there's a different problem. The exception is
different if we feed different data to the function. So, let's try
again, but maybe we catch the TypeError instead.
def catchTypeError(val, default=0):
'''somebody passed in a non-convertible type, return a default'''
try:
return int(val)
except TypeError:
return default
This seems to work--but, now, if we try our other values (which
worked before), they don't work now:
>>> catchTypeError(dict())
0
>>> catchTypeError(0)
0
>>> catchTypeError('-3')
-3
>>> catchTypeError('Catullus')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in catchTypeError
ValueError: invalid literal for int() with base 10: 'Catullus'
Step 3: catch several different classes of Exception
----------------------------------------------------
You can a single try-except block to catch multiple classes of
specific Exception (and then perform exception handling).
def catchTVError(val, default=0):
'''somebody passed in a non-convertible type, return a default'''
try:
return int(val)
except (ValueError, TypeError):
return default
In this little toy demonstration, you can see that this is probably
the desired functionality.
>>> catchTVError(dict())
0
>>> catchTVError(0)
0
>>> catchTVError(-3)
-3
>>> catchTVError('Catullus')
0
Step 4: catch many different classes of Exception
-------------------------------------------------
Now, suppose you want to deal with each possible exception in a different
manner....you can have different exception-handling behaviour for each class of
exception.
def altCatchTVError(val, default=42):
'''somebody passed in a non-convertible type, return a default'''
try:
return int(val)
except ValueError:
return abs(default)
except TypeError:
return 0 - abs(default)
As you can see, this offers a good deal more flexibility for handling specific
exceptions that you might encounter.
>>> altCatchTVError(0)
0
>>> altCatchTVError(22)
22
>>> altCatchTVError('17')
17
>>> altCatchTVError('-3')
-3
>>> altCatchTVError('str')
42
>>> altCatchTVError(dict())
-42
Interlude and recommendation
----------------------------
As you can see, there are many possible Exceptions that can be raised when you
are calling a simple builtin function, int().
Consider now what may happen when you call out to a different program; you
indicated that your run() function calls out to subprocess.Popen(). There are
many more possible errors that can occur, just a few that can come to my mind:
* locating the program on disk
* setting up the file descriptors for the child process
* fork()ing and exec()ing the program
* memory issues
* filesystem disappears (network goes away or block device failure)
Each one of these possible errors may translate to a different exception. You
have been tempted to do:
try:
run()
except:
pass
This means that, no matter what happens, you are going to try to keep
continuing, even in the face of massive failure.
To (#1) improve the safety of your program and the environments in
which it operates, to (#2) improve your defensive programming
posture and to (#3) avoid frustrating your own debugging at some
point in the future, you would be well-advised to identify which
specific exceptions you want to ignore.
As you first try to improve the resilience of your program, you may
not be certain which exceptions you want to catch and which
represent a roadblock for your progam. This is something
that usually comes with experience.
To get that experience you can define your own exception (it'll
never get raised unless you raise it, so do not worry). Then,
create your try-except block to catch only that one. As you
encounter other exception that you are certain you wish to handle,
you can do something with them:
class UnknownException(Exception):
pass
def prep_host():
"""
Prepare clustering
"""
for cmd in ["ls -al",
"touch /tmp/file1",
"mkdir /tmp/dir1"]:
try:
if not run_cmd_and_verify(cmd, timeout=3600):
logging.info("Preparing cluster failed ...")
return False
except (UnknownException,):
pass
logging.info("Preparing Cluster.....Done !!!")
return True
Now, as you develop your program and encounter new exceptions, you
can add new except clauses to the above block with appropriate
handling, or (re-)raising the caught exception.
Comments on shelling out to other programs and using exceptions
---------------------------------------------------------------
Exceptions are great for catching logic errors, type errors,
filesystem errors and all manner of other errors within Python
programs and runtime environments. You introduce a significant
complexity the moment you fork a child (calling subprocess.Popen).
It is good, though, that you are testing the return code of the
cmd that you pass to the run() function.
Final advice:
-------------
Do not use a bare try-except.
You will frustrate your own debugging and your software may end up
trying to excecute code paths (or external programs, as you are
doing right now) for which you were sanity checking.
-Martin
--
Martin A. Brown
http://linux-ip.net/
--
https://mail.python.org/mailman/listinfo/python-list
