Re: using python interpreters per thread in C++ program
> Did you remeber to acquire the GIL? The GIL is global to the process No, I did not use GIL. -- Why do we need to use GIL even though python is private to each thread? -- For using GIL, do we need to initialize GIL at startup and destroy/ finalize it at end? -- With GIL, we are not achieiving concurrent operations on python interpreters across threads. When it comes to calling Py-C APIs, program is working like in a single threaded mode. Is there any way, we can avoid the GIL locking, etc? Please guide. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: using python interpreters per thread in C++ program
> Did you remeber to acquire the GIL? The GIL is global to the process > (hence the name). No, I did not use GIL. -- For using GIL, do we need to initialize GIL at startup and destroy/ finalize it at end? -- Are there any configuration & build related flags that I need to use to make this work? Please guide. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: using python interpreters per thread in C++ program
On Sep 7, 2:04 pm, sturlamolden wrote:
> I just showed you how...
Modified the thread function to use these APIs, but the call to
PyGILState_Ensure() is not returning at all.
void *callPyFunction(void * arg)
{
// Method two to get function eval
long thridx=(long)arg;
printf("\n>my num=%d, calling showstr pyfunction\n",thridx);
char callpyf[] = "showstr(100)\n";
PyGILState_STATE state = PyGILState_Ensure();
printf("after PyGILState_Ensure\n");
PyObject* pycall = PyRun_String(callpyf,Py_file_input,glb, loc);
if(pycall == NULL || PyErr_Occurred()){
printf("PyRun_String failed\n");
PyErr_Print();
} else
printf("%d thread called showstr pyfunction ok\n",thridx);
PyGILState_Release(state);
pthread_exit(NULL);
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: using python interpreters per thread in C++ program
On Sep 7, 3:41 pm, Graham Dumpleton wrote: > On Sep 7, 3:42 pm, sturlamolden wrote: > interpreters. The simplified GIL state API you mentioned only works > for threads operating in the main (first) interpreter created within > the process. I modified my program to have Py_Initialize and compilation of one Python function done in main() thread. Then I am calling only that function in callPyFunction() thread. But, this thread does not come out of PyGILState_Ensure() function. > The OP can do what they want, but they need to user lower level > routines for creating their own thread state objects and acquiring the > GIL against them. > > Graham What are the "lower level routines" for creating own thread state objects & acquiring GILs. Also, where can I find more information about those routines? Please guide. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: using python interpreters per thread in C++ program
Actually, I modified my program to have a single shared Py-interpreter across all threads to test the usage of GIL. So, I did Py_Initialize in main() function and only called that python function in different threads. But this is not the way I want to use interpreters in my code. I am looking for using sub-interpreters, only that I did not know this this term, till you mentioned it here. So bvefore this, I was calling py_Initialize in each of the C++ level threads, which was wrong. I should be using Py_NewInterpreter() in my threads and Py_Initialize () in main() thread. Please correct me if I am wrong. not something you want to do unless you want to understand Python C API internals very well. Are these APIs really tricky to use with C++ embedding? Can't they be implemented thru C APIs? I need to use these to get the proper concurrency in my multi-threaded application without any synchronization mechanisms. -- http://mail.python.org/mailman/listinfo/python-list
Re: using python interpreters per thread in C++ program
My application is a TCP server having multiple client connectons. C++ PTHREADS are for each connected socket and the message received on the socket is evaluated by python functions. If I use only one process level python interpreter, then every thread has to lock the GIL & so blocking the other threads from executing the python code even if it is not the same python function that locking thread is calling. -- That's why I tried using python interpreters per thread. But that also required GIL locking & so cannot be used. -- I cannot use python threads inside the Pyton intrepreter because then I will have to have some mechanism for communiaction between C++ Pthreads with these python threads. I think there is no way that we can achieve this because of the GIL being a process level state. Least I can do is have one python interpreter initialized in main thread and lock the GIL in every thread for python calls. -- http://mail.python.org/mailman/listinfo/python-list
Re: using python interpreters per thread in C++ program
On Sep 8, 2:46 pm, I V wrote: > Do you have to use threads? If you use a process per connection, rather > than a thread, each process will have its own GIL. No, i cannot change from threads to processes for handling connections. This will change the complete design of our application which is not feasilbe for python evaluation of the strings. -- http://mail.python.org/mailman/listinfo/python-list
Python API
Hi What are all the python api, u used in your python programming, we used more api but may we forgot those, so i just want to list down the api we familiar aboutplease add your replies... -- http://mail.python.org/mailman/listinfo/python-list
How do I check all variables returned buy the functions exists
I have a function that return's x variables How do I check if all the the values returned are not None/False/0/'' Here is the same program to demonstrate this , but I felt this can be better any suggestions ? # vi file.py import random import string def return_x_values(): " returns x strings for further processing" value1 = random.choice(string.ascii_letters) value2 = random.choice(string.ascii_letters) value3 = random.choice(string.ascii_letters) return (value1, value2, value3) #unpack them value1, value2 , value3 = return_x_values() # check if its not none # I think this can be better if value1 and value2 and value3 : print "continue with the program" else: print "Exting the program" # python file.py continue with the program I am a Linux user with Python 2.7 Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
How to ingore "AttributeError: exception
I have two possible values for Z_block in the block code i.e disk_object.data.data.di_data[0] or block.data.data.di_data.data[0][0] def get_block(): ''' Get Z block '' if block.data.data.di_data.data[0][0] is not None: Z_block = block.data.data.di_data.data[0][0] else: Z_block = disk_object.data.data.di_data[0] if not Z_block: return False return Z_block I have a problem with if and else satement i.e if IF codition fails the code would exit with "AttributeError: 'list' object has no attribute 'data' " error Any suggestion on how this can be handled better , Will ignoring the exceptions in try -except with pass be good or are there easier ways ) , try: Z_block = block.data.data.di_data.data[0][0]except AttributeError as e: pass I am a Linux user on Python 2.7 Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Re: How to ingore "AttributeError: exception
> > > is a perfectly good pattern to use. > Thanks looks nice :) > > > > > > > > I am a Linux user on Python 2.7 > > Have you considered moving to Python 3? > Not yet , but Is there something that I need to consider in the current context? Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
How to join elements at the beginning and end of the list
How to join each elements with a delimiter at (1) beginning and end of the list and (2) connecting all elements of the list Example : >>> value_list = [1, 2, 3, 4, 56, 's'] I want this to be converted in this from '||1||2||3||4||56||s||' Here is my solution >>> values = '||' + '||'.join(map(str, value_list)) + '||' >>> values '||1||2||3||4||56||s||' Iam joining the elements at the beginning and end of the list using '+' operator any other solution, this is not looking neater I am a Linux user using python 2.7 Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Helloworld with Python C extension
Hello Team ,
I need you input on the below hello world program. I a m trying to add a
python binding which will return the character for the given index . I am
on Python 2.7 and linux
Example :
>> string ='helloworld'
>>dda_hello(5)
>> 'w'
/*
+ * Hello world example for python bindings
+ */
+
+static char* string = "helloworld";
+char dda_hello(int i)
+ {
+ return string[i];
+ }
+
+static PyObject *
+py_dda_hello(PyObject *self, PyObject *args )
+{
+ int index;
+ char char1;
+ if (!PyArg_ParseTuple(args, "i", &index))
+ return NULL;
+ char1 = dda_hello(index);
+ return Py_BuildValue("s",char1);
+}
+
+/*
@@ -1674,6 +1705,10 @@ PyMethodDef xyz_methods[] = {
+{"dda_hello", py_dda_hello, METH_VARARGS,
+"Returns the character entered for a given index"},
>>> import as.ds.dss as daa
>>> print dda.dda_hello(1)
zsh: segmentation fault (core dumped) python
Apologies for posting the diff , I didn't find a better way
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Helloworld with Python C extension
>
>
>
> Py_BuildValue with an "s" expects a C string - that is, a pointer to
> char, not just a single character. You'd need to do something like
> this:
>
> char buf[2] = {char1, 0};
> return Py_BuildValue("s", buf);
>
> ChrisA
Thanks Chris for the clue's it worked, I was just wondering how
could the
C extension be debugged ?
We have pdb at python side and gdb for C , can we run gdb on python side
? if there is a crash like the one we saw in the above diff we are
clueless of what's happening? any idea or recommendation on how we handle
such cases
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
C Python extension to export an Function
Hi Team,
Iam on python 2.7 and Linux.
Iam pretty new to C Python extension , I was able to export few simple
modules to python and it look like the cool thing to do , but Iam stuck
for with a problem now , Iam not able to figure out how to export
fun_addr_from_addr()
to Python. I would need some suggestion on how to develop a C Python
API for the below piece of code.
1. The end goal is to export fun_addr_from_addr(baddr, *baddr) to Python.
Perhaps by creating a python wrapper for fun_bh_toblkoff() would help me ?
Here is the sample example code.
fun_toblkoff(dda_locn_t *locn,
struct ifs_baddr *baddr, int *poff, int *psize)
{
if (dda_locnparm_get(locn) != OK)
return FAIL;
/* fill in baddr */
*baddr = dda_index_to_baddr(locn->idx);
if (locn->flags & DLFLAG)
locn->xoff = fun_addr_from_addr(baddr, *baddr);
*poff = 0;
*psize = _BSIZE;
return OK;
}
So the primary challenge when writing a wrapper for a single
function is figuring out how to get the arguments from
Python form to C form, and how to get the result back to Python form.
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: C Python extension to export an Function
On Thu, Sep 1, 2016 at 12:32 PM, dieter wrote:
> Ganesh Pal writes:
>
> > Iam pretty new to C Python extension , I was able to export few simple
> > modules to python and it look like the cool thing to do ...
>
> Maybe, it is a good idea to have a look at "cython".
>
> "cython" is a compiler. It translates Python code enhanced with
> special annotations into C. The annotations mostly tell the compiler
> that something ("object", "method", "function", ...) should be at "C"
> rather than "Python" level, thus avoiding much of Python's overhead
> and allows to do things possible in "C" but not in "Python".
>
>
> Developing safe "C" extensions for Python is difficult. You
> need some quite deep understanding of the Python-C interface
> and must be very careful to observe all requirements (especially
> those related to proper reference management).
>
> Developing "C" extensions with "cython" is much easier as
> "cython" hides many of the complexities and takes care of most
> requirements.
>
Dear Dieter ,
Really appreciate the reply and your suggestion on trying to use "cython"
, but my whole idea of using "C" extension is to regular C codes . We
have bunch of C code that's already available and C -Python seems to suit
me better
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: C Python extension to export an Function
Thanks stefan and Gollwitzer , good to know there are many ways to do this
i.e via cython or SWIG but the C/Python API
<https://docs.python.org/2/c-api/> is probably the most widely used method
- not for it’s simplicity but for the fact that you can manipulate python
objects in your C code.
I want to use C/Python API <https://docs.python.org/2/c-api/>
On Thu, Sep 1, 2016 at 6:32 PM, Stefan Behnel wrote:
> Ganesh Pal schrieb am 01.09.2016 um 14:30:
> > On Thu, Sep 1, 2016 at 12:32 PM, dieter wrote:
> >> Ganesh Pal writes:
> >>> Iam pretty new to C Python extension , I was able to export few simple
> >>> modules to python and it look like the cool thing to do ...
> >>
> >> Maybe, it is a good idea to have a look at "cython".
> >>
> >> "cython" is a compiler. It translates Python code enhanced with
> >> special annotations into C. The annotations mostly tell the compiler
> >> that something ("object", "method", "function", ...) should be at "C"
> >> rather than "Python" level, thus avoiding much of Python's overhead
> >> and allows to do things possible in "C" but not in "Python".
> >>
> >> Developing safe "C" extensions for Python is difficult. You
> >> need some quite deep understanding of the Python-C interface
> >> and must be very careful to observe all requirements (especially
> >> those related to proper reference management).
> >>
> >> Developing "C" extensions with "cython" is much easier as
> >> "cython" hides many of the complexities and takes care of most
> >> requirements.
> >
> > Really appreciate the reply and your suggestion on trying to use
> "cython"
> > , but my whole idea of using "C" extension is to regular C codes . We
> > have bunch of C code that's already available and C -Python seems to
> suit
> > me better
>
> From your response it's not obvious whether you are aware that Cython also
> makes it substantially easier to *interface* CPython with external C code,
> in the same way that it makes it easy (but not necessary) to *avoid*
> writing C in the first place. So I thought I'd just mention that this is
> not a reason to rule it out as an excellent option.
>
> Stefan
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
list or dictionary
I am on python 2.7 and Linux
I have the stdout in the below form , I need to write a function to get
hostname for the given id.
Example:
>>> stdout
'hostname-1 is array with id 1\nhostname-2 is array with id 2\nhostname-3
is array with id 3\n'
def get_hostname(id)
return id
what's a better option here
1. store it in list and grep for id and return
2. store it in dict as key and value i.e hostname = { 'hostname1': 1} and
return key
3. any other simple options.
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: list or dictionary
Thanks Steve for the clues , quickly tried out # Version 1 doesn't seen
to work.
>>> for line in hostname:
... regex = r'(.*) is array with id {}'.format(devid)
... mo = re.search(regex, line)
... print line, regex, mo
... if mo is not None:
... print mo.group(1)
...
RX-145-1 is array id 1 (.*) is array with id 3 None
RX-145-2 is array id 2 (.*) is array with id 3 None
RX-145-3 is array id 3 (.*) is array with id 3 None
>>> hostname
['RX-145-1 is array id 1', 'RX-145-2 is array id 2', 'RX-145-3 is array id
3']
>>> type(devid)
>>> devid
3
---
>>> devid = '3'
>>> for line in hostname:
... regex = r'(.*) is array with id {}'.format(devid)
... mo = re.search(regex, line)
... print line, regex, mo
... if mo is not None:
...print mo.group(1)
...
RX-145-1 is array id 1 (.*) is array with id 3 None
RX-145-2 is array id 2 (.*) is array with id 3 None
RX-145-3 is array id 3 (.*) is array with id 3 None
>>> type(devid)
--
>>> for line in hostname:
... regex = r'(.*) is array with id %d' % (devid)
... mo = re.search(regex, line)
... print line, regex, mo
... if mo is not None:
...print mo.group(1)
...
RX-145-1 is array id 1 (.*) is array with id 3 None
RX-145-2 is array id 2 (.*) is array with id 3 None
RX-145-3 is array id 3 (.*) is array with id 3 None
---
Looks like Iam missing something ?
Regards,
Ganesh
On Wed, Sep 21, 2016 at 5:57 AM, Steve D'Aprano
wrote:
> On Wed, 21 Sep 2016 04:04 am, Ganesh Pal wrote:
>
> > I am on python 2.7 and Linux
> >
> > I have the stdout in the below form , I need to write a function to get
> > hostname for the given id.
> >
> >
> > Example:
> >
> >>>> stdout
> > 'hostname-1 is array with id 1\nhostname-2 is array with id
> 2\nhostname-3
> > is array with id 3\n'
> >
> >
> > def get_hostname(id)
> >return id
> >
> > what's a better option here
> >
> > 1. store it in list and grep for id and return
> > 2. store it in dict as key and value i.e hostname = { 'hostname1': 1} and
> > return key
> > 3. any other simple options.
>
>
> Why don't you write a function for each one, and see which is less work?
>
> # Version 1: store the hostname information in a list as strings
>
> hostnames = ['hostname-1 is array with id 1',
> 'hostname-2 is array with id 2',
> 'hostname-842 is array with id 842']
>
> def get_hostname(id):
> for line in hostnames:
> regex = r'(.*) is array with id %d' % id
> mo = re.match(regex, line)
> if mo is not None:
> return mo.group(1)
> raise ValueError('not found')
>
>
> # Version 2: store the hostname information in a dict {hostname: id}
>
> hostnames = {'hostname1': 1, 'hostname2': 2, 'hostname842': 842}
>
> def get_hostname(id):
> for key, value in hostnames.items():
> if value == id:
> return key
> raise ValueError('not found')
>
>
> # Version 3: use a dict the way dicts are meant to be used
>
> hostnames = {1: 'hostname1', 2: 'hostname2', 842: 'hostname842'}
>
> def get_hostname(id):
> return hostnames[id]
>
>
>
>
>
>
>
>
> --
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: list or dictionary
Thanks , and it has to be re.match()
On Thu, Sep 22, 2016 at 12:18 AM, MRAB wrote:
> On 2016-09-21 19:35, Ganesh Pal wrote:
>
>> Thanks Steve for the clues , quickly tried out # Version 1 doesn't seen
>> to work.
>>
>>
>> for line in hostname:
>>>>>
>>>> ... regex = r'(.*) is array with id {}'.format(devid)
>> ... mo = re.search(regex, line)
>> ... print line, regex, mo
>> ... if mo is not None:
>> ... print mo.group(1)
>> ...
>> RX-145-1 is array id 1 (.*) is array with id 3 None
>> RX-145-2 is array id 2 (.*) is array with id 3 None
>> RX-145-3 is array id 3 (.*) is array with id 3 None
>>
>>> hostname
>>>>>
>>>> ['RX-145-1 is array id 1', 'RX-145-2 is array id 2', 'RX-145-3 is array
>> id
>> 3']
>>
>>> type(devid)
>>>>>
>>>>
>>
>>> devid
>>>>>
>>>> 3
>>
>> ---
>>
>>> devid = '3'
>>>>> for line in hostname:
>>>>>
>>>> ... regex = r'(.*) is array with id {}'.format(devid)
>> ... mo = re.search(regex, line)
>> ... print line, regex, mo
>> ... if mo is not None:
>> ...print mo.group(1)
>> ...
>> RX-145-1 is array id 1 (.*) is array with id 3 None
>> RX-145-2 is array id 2 (.*) is array with id 3 None
>> RX-145-3 is array id 3 (.*) is array with id 3 None
>>
>>> type(devid)
>>>>>
>>>>
>>
>> --
>>
>>> for line in hostname:
>>>>>
>>>> ... regex = r'(.*) is array with id %d' % (devid)
>> ... mo = re.search(regex, line)
>> ... print line, regex, mo
>> ... if mo is not None:
>> ...print mo.group(1)
>> ...
>> RX-145-1 is array id 1 (.*) is array with id 3 None
>> RX-145-2 is array id 2 (.*) is array with id 3 None
>> RX-145-3 is array id 3 (.*) is array with id 3 None
>>
>> ---
>>
>> Looks like Iam missing something ?
>>
>> [snip]
> The lines have "array id", but the regex has "array with id".
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
How to you convert list of tuples to string
Dear friends , I am using fedora 18 and on Python 2.7 version I have a list of tuples as shown below >> list [(1, 1, 373891072L, 8192), (1, 3, 390348800L, 8192), (1, 4, 372719616L, 8192), (2, 3, 382140416L, 8192), (2, 5, 398721024L, 8192), (3, 1, 374030336L, 8192), (3, 3, 374079488L, 8192), (3, 5, 340058112L, 8192)] (a) I need to select any element randomly the list say (x, y, xL, 8192) >>> list [(1, 1, 373891072L, 8192), (1, 3, 390348800L, 8192), (1, 4, 372719616L, 8192), (2, 3, 382140416L, 8192), (2, 5, 398721024L, 8192), (3, 1, 374030336L, 8192), (3, 3, 374079488L, 8192), (3, 5, 340058112L, 8192)] >>> import random >>> i = random.randrange(len(list)) >>> sel_item = list[i] >>> sel_item (3, 5, 340058112L, 8192) (b) Then convert the selected item in the below format i.e 1,1,373891072:8192 ( strip L and add :) >>> sel_item (3, 5, 340058112L, 8192) >> c1 = ','.join(map(str,sel_item)) # what happened to 'L' it got stripped automatically ? will these be a problem >>> c1 '3,5,340058112,8192' #last four are always 8912 and >>> c1 = c1[0:-5] + ':8912' >>> c1 '3,5,340058112:8912' >>> Any better suggestion to improve this piece of code and make it look more / pythonic Regards, Ganesh Pal -- https://mail.python.org/mailman/listinfo/python-list
correct way to catch exception with Python 'with' statement
Dear Python friends,
Any suggestion on how to add exception and make the below program look
better , I am using Python 2.7 and Linux
def create_files_append():
""" """
try:
os.makedirs(QA_TEST_DIR)
except:
raise OSError("Can't create directory (%s)!" % (QA_TEST_DIR))
# Create few files and write something
for i in range(1000):
with open(os.path.join(QA_TEST_DIR,"filename%d" %i),'w') as f:
f.write("hello")
# Append the files
for i in range(1000):
with open(os.path.join(QA_TEST_DIR,"filename%d" %i),'w') as f:
f.write("hello")
return True
---
What will be the best way to catch the exception in the above program ?
Can we replace both the with statement in the above program with something
like below
try:
for i in range(1000):
with open(os.path.join(QA_TEST_DIR,"filename%d" %i),'w') as f:
f.write("hello")
except IOError as e:
raise
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Simple Python equivalent for the shell command
I was trying to write a function that will return me the unique number
associated with each employee id.The command has the output in the below
pattern
Linux-Box-1# employee_details ls
List of names:
100910bd9 s7018
100d60003 s7019
110610bd3 s7020
100d60002 s7021
Linux-Box-1# employee_details ls | grep "s7020" | awk '{print $1}'
100d60003
It's a one liner in Shell :)
I tried converting the same in the python style , Any better suggestion
and loop holes in the below program
def get_unique_number(str(emp_id)):
""" Return the unique number associated with each employee id """
out, err, rc = run("employee_details ls", timeout=600)
emp_unum=""
if rc != 0:
return False
for line in out.split('\n'):
if emp_id in line:
emp_unum = line.split()[0]
return emp_unum
I am on Python 2.7 and Linux OS
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: correct way to catch exception with Python 'with' statement
On Mon, Nov 28, 2016 at 1:16 PM, Steven D'Aprano < steve+comp.lang.pyt...@pearwood.info> wrote: > > > There is no need to return True. The function either succeeds, or it > raises an > exception, so there is no need to return any value at all. > > I returned True here ,because based on the result of this function , I would want to perform next steps Example if create_files_append(): do_somthing() else: do_next_thing() > > Your comment says "append the files", but you're not appending to the > files, > you are overwriting them. So your code is better written like this: > > Yes , correct and apologies there was a typo it should have been 'a' instead of 'w' . Thanks for the comments -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple Python equivalent for the shell command
I remembered that I might need to add an else condition if the emp_num
does not exist , so re sending the updated code
def get_unique_number(str(emp_id)):
""" Return the unique number associated with each employee id """
out, err, rc = run("employee_details ls", timeout=600)
emp_unum=""
if rc != 0:
return False
for line in out.split('\n'):
if emp_id in line:
emp_unum = line.split()[0]
else:
print("emp_unum does not exist")
return False
return emp_unum
PS : [Edited the above code with else condition]
Regards,
Ganesh
On Mon, Nov 28, 2016 at 8:38 PM, Ganesh Pal wrote:
>
>
> I was trying to write a function that will return me the unique number
> associated with each employee id.The command has the output in the below
> pattern
>
> Linux-Box-1# employee_details ls
> List of names:
> 100910bd9 s7018
> 100d60003 s7019
> 110610bd3 s7020
> 100d60002 s7021
>
>
> Linux-Box-1# employee_details ls | grep "s7020" | awk '{print $1}'
> 100d60003
>
> It's a one liner in Shell :)
>
>
> I tried converting the same in the python style , Any better suggestion
> and loop holes in the below program
>
> def get_unique_number(str(emp_id)):
> """ Return the unique number associated with each employee id """
> out, err, rc = run("employee_details ls", timeout=600)
> emp_unum=""
> if rc != 0:
> return False
>
> for line in out.split('\n'):
> if emp_id in line:
>emp_unum = line.split()[0]
> return emp_unum
>
> I am on Python 2.7 and Linux OS
>
> Regards,
> Ganesh
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: correct way to catch exception with Python 'with' statement
Thanks Steve I got what you were trying to explain , nice learning from this conversation , what I was really doing wrong I had broken down my huge code into a simple program and had missed out returning False. On Tue, Nov 29, 2016 at 11:01 AM, Steven D'Aprano < steve+comp.lang.pyt...@pearwood.info> wrote: > On Tuesday 29 November 2016 02:18, Ganesh Pal wrote: > > > On Mon, Nov 28, 2016 at 1:16 PM, Steven D'Aprano < > > steve+comp.lang.pyt...@pearwood.info> wrote: > > > >> > >> > >> There is no need to return True. The function either succeeds, or it > >> raises an > >> exception, so there is no need to return any value at all. > >> > >> > > I returned True here ,because based on the result of this function , > > But the function *always* returns True, or it doesn't return at all: it > raises. > > Unless you have something like: > > def func(): >do some work >if condition: > return False >do more work >return True > > or similar, there's no point. When you write the documentation for the > function, if it can only ever return True, then don't worry about returning > True. Take the built-in methods as an example: dict.update either > succeeds, or > it raises an exception. It doesn't return True: > > # this is unnecessary > flag = mydict.update(another_dict) > if flag: > print "update succeeded" > else: > print "update failed" > > > That cannot happen, because if the update fails, an exception is raised. > > The bottom line is, since your function *only* has "return True" and > doesn't > have "return False" anywhere, there is no point to the "return True." > > > > I would want to perform next steps > > > > Example > > if create_files_append(): > >do_somthing() > > else: > > do_next_thing() > > That cannot happen. It either returns True, or it raises an exception, so > the > "else" clause will not be executed. > > > > > -- > Steven > "Ever since I learned about confirmation bias, I've been seeing > it everywhere." - Jon Ronson > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple code and suggestion
On Wed, Nov 30, 2016 at 7:33 PM, Dennis Lee Bieber
wrote:
> On Wed, 30 Nov 2016 18:56:21 +0530, g thakuri
> declaimed
> the following:
>
> >Dear Python friends,
> >
> >I have a simple question , need your suggestion the same
> >
> >I would want to avoid using multiple split in the below code , what
> options
> >do we have before tokenising the line?, may be validate the first line any
> >other ideas
> >
> > cmd = 'utility %s' % (file)
> > out, err, exitcode = command_runner(cmd)
> > data = stdout.strip().split('\n')[0].split()[5][:-2]
> >
> 1) Where did "stdout" come from? (I suspect you meant just
> "out")
>
My bad it should have been out , here is the updated code
> cmd = 'utility %s' % (file)
> out, err, exitcode = command_runner(cmd)
> data = out.strip().split('\n')[0].split()[5][:-2]
>
> 2) The [0] indicates you are only interested in the FIRST
> LINE; if so,
> just remove the entire ".split('\n')[0]" since the sixth white space
> element on the first line is also the sixth white space element of the
> entire returned data.
>
>
Yes , I am interested only in the first line , may be we can test if we
have a line[0] before tokenising the line ?
>
>
--
https://mail.python.org/mailman/listinfo/python-list
how do I retry a command only for a specific exception / error
import time
from functools import wraps
from qa.utils.easy_popen import run
def retry(ExceptionToCheck, tries=4, delay=5, backoff=2, logger=None):
""" Retry calling the decorated function """
def deco_retry(f):
@wraps(f)
def f_retry(*args, **kwargs):
mtries, mdelay = tries, delay
while mtries > 1:
try:
return f(*args, **kwargs)
except ExceptionToCheck, e:
msg = "%s, Retrying in %d seconds..." % (str(e), mdelay)
if logger:
logger.warning(msg)
else:
print msg
time.sleep(mdelay)
mtries -= 1
mdelay *= backoff
return f(*args, **kwargs)
return f_retry # true decorator
return deco_retry
@retry(Exception, tries=4, delay=2)
def test_network(text):
try:
cmd = "ifconfig -a"
stdout, stderr, exit_code = run(cmd, timeout=300)
print stdout, stderr, exit_code
if exit_code != 0:
raise RuntimeError("ERROR (exit_code %d): "
"\nstdout: %s\nstderr: %s\n" % (exit_code,
stdout, stderr))
except Exception as e:
print str(e)
raise Exception("Failed")
print "Success: ", text
test_network("it works!")
All that I am trying to do here is write a generic function that will
re-retry
the command few more times before failing the test
Case 1: + ve case ( say cmd= ifconfig –a )
gpal-zkj2wrc-1# python file5.py
vmx0: flags=1008843
metric 0 mtu 1500
options=403bb
ether 00:50:56:90:ef:3d
inet 1.224.39.1 netmask 0x broadcast 1.224.255.255 zone 1
inet6 fe80::250:56ff:fe90:ef3d%vmx0 prefixlen 64 scopeid 0x1 zone 1
inet6 fdfe:9042:c53d:0:250:56ff:fe90:ef3d prefixlen 64 zone 1
nd6 options=21
media: Ethernet 10Gbase-T
status: active
vmx1: flags=8843 metric 0 mtu 1500
options=403bb
ether 00:50:56:90:62:44
inet 10.224.39.1 netmask 0xfc00 broadcast 10.224.39.255 zone 1
nd6 options=29
media: Ethernet 10Gbase-T
status: active
Case 1 : Say I have invalid input command ( say ifconfig –a ) it will work
gpal-zkj2wrc-1# python file5.py
ifconfig: illegal option -- z
usage: ifconfig [-f type:format] [-L] [-C] [-g groupname] interface
address_family
[address [dest_address]] [parameters]
ifconfig interface create
ifconfig -a [-L] [-C] [-g groupname] [-d] [-m] [-u] [-v]
[address_family]
ifconfig -l [-d] [-u] [address_family]
ifconfig [-L] [-C] [-g groupname] [-d] [-m] [-u] [-v]
1
ERROR (exit_code 1):
stdout:
stderr: ifconfig: illegal option -- z
usage: ifconfig [-f type:format] [-L] [-C] [-g groupname] interface
address_family
[address [dest_address]] [parameters]
ifconfig interface create
ifconfig -a [-L] [-C] [-g groupname] [-d] [-m] [-u] [-v]
[address_family]
ifconfig -l [-d] [-u] [address_family]
ifconfig [-L] [-C] [-g groupname] [-d] [-m] [-u] [-v]
Failed, Retrying in 2 seconds...
ifconfig: illegal option -- z
usage: ifconfig [-f type:format] [-L] [-C] [-g groupname] interface
address_family
[address [dest_address]] [parameters]
ifconfig interface create
ifconfig -a [-L] [-C] [-g groupname] [-d] [-m] [-u] [-v]
[address_family]
ifconfig -l [-d] [-u] [address_family]
ifconfig [-L] [-C] [-g groupname] [-d] [-m] [-u] [-v]
1
ERROR (exit_code 1):
stdout:
stderr: ifconfig: illegal option -- z
usage: ifconfig [-f type:format] [-L] [-C] [-g groupname] interface
address_family
[address [dest_address]] [parameters]
ifconfig interface create
ifconfig -a [-L] [-C] [-g groupname] [-d] [-m] [-u] [-v]
[address_family]
ifconfig -l [-d] [-u] [address_family]
ifconfig [-L] [-C] [-g groupname] [-d] [-m] [-u] [-v]
Failed, Retrying in 4 seconds...
Case 3: Assuming my command threw an exception say OSError , how do I
retry a command only for a specific exception / error
I am on Python 2.7 and Linux
Regards,
Ganesh Pal
--
https://mail.python.org/mailman/listinfo/python-list
Re: how do I retry a command only for a specific exception / error
On Fri, Mar 16, 2018 at 11:21 AM, Steven D'Aprano <
steve+comp.lang.pyt...@pearwood.info> wrote:
> On Fri, 16 Mar 2018 11:04:22 +0530, Ganesh Pal wrote:
>
> > All that I am trying to do here is write a generic function that will
> > re-retry
> > the command few more times before failing the test
>
>
> Something like this should do it. It gives up immediately on fatal errors
> and tries again on temporary ones. (You have to specify what you consider
> fatal or temporary, of course.) It uses exponential backup to wait longer
> and longer each time you fail, before eventually giving up.
>
>
> This is a good suggestion , I like the way this is written , but what I
> have failed to figure out is how to translate the possible
> TemporaryFailureErrors
> to a different exception class/type and retry .
>
> In my case , every command is executed using a run() function that
> calls out to subprocess.Popen(). Which will return stdout, stderr,
> exit_code and we would need to retry only for a specific
> TemporaryFailureError .
>
>
>
> Example : Say , If we are not able to SSH to the host , and I get
> “connection refused” error I would want to retry only for this specific case
>
>
>
> # Sample modified code
>
>
>
> delay = 2
>
> max_attempts =4
> for attempts in range(max_attempts):
> try:
> cmd = "ssh r...@localhost.com"
>
> stdout, stderr, exit_code = run(cmd, timeout=300)
>
> print stdout, stderr, exit_code
>
> if exit_code != 0:
>
> raise RuntimeError("ERROR (exit_code %d): "
>
>"\nstdout: %s\nstderr: %s\n" % (exit_code,
> stdout, stderr))
>
> except Exeception as e :
> raise
>
> # if we have “connection refused” error then retry after some time
> except TemporaryFailureError:
> sleep(delay)
> delay *= 2
> else:
> break
> else:
> raise RuntimeError("too many attempts")
>
>
>
>
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: how do I retry a command only for a specific exception / error
Please ensure quoted text is quoted, and new text you write is unquoted. > That way you are more likely to get useful > Sorry , Steve I didn't realize but thanks for pointing out I will take care I was on a mobile phone and messed the quoted text >Something like this should do it. It gives up immediately on fatal >errors >and tries again on temporary ones. (You have to specify what you >consider >fatal or temporary, of course. This is a good suggestion , I like the way the code is written , but what I have failed to understand is how to translate the possible TemporaryFailureErrors to a different exception class/type and retry . In my case , every command is executed using a run() function that calls out to subprocess.Popen(). Which will return stdout, stderr, exit_code and we would need to retry only for a specific TemporaryFailureError . Example : Say , If we are not able to SSH to the host , and I get “connection refused” error I would want to retry only for this specific case -- https://mail.python.org/mailman/listinfo/python-list
String Formatting with new .format()
Hi Team,
Just a quick suggestion, on string formatting with .format() which of the
below is better , given both give the same result .
>>> attempts = 1
>>> msg2 = "Hello"
>>> print "Retry attempt:{0} for error:{1}".format(attempts,msg2)
Retry attempt:1 for error:Hello
OR
>>> attempts = 1
>>> msg2 = "Hello"
>>> print "Retry attempt:{0} for error:{0}".format(attempts,msg2)
Retry attempt:1 for error:1
>>>
PS : This is the silly question but I wanted to know if it really makes any
difference
I am on Python 2.7 and Linux
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: String Formatting with new .format()
> > > Or maybe they're not giving the same result. I'm a little confused here. > > Thanks Chris, for the reply they appear to give the same result . -- https://mail.python.org/mailman/listinfo/python-list
Pep8 for long pattern
Hello Python friends, How do I split the below regex , so that it fits within the character limit of 79 words pattern = [ r'(?P([0-9a-fA-F]+:[0-9a-fA-F]+:[0-9a-fA-F]+:[0-9a-fA-F]+:[0-9a-fA-F]+::HEAD))', r'(?P(owner:\s+[0-9a-fA-F]+:[0-9a-fA-F]+:[0-9a-fA-F]+:[0-9a-fA-F]+:[0-9a-fA-F]+::HEAD))', '.'] I am using Python 2.7 + Linux Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Re: String Formatting with new .format()
>
> Or maybe they're not giving the same result. I'm a little confused here.
>
My Bad and Apologies , I should be fined for pasting wrong question.
Actually I wanted to know if its ok to use just empty {} with .format()
or use {} with values i.e {0} {1} both will give the same results anyway
Example:
>>> attempts =1
>>> msg2 = "Hello"
>>> print "Retry attempt:{} for error:{}".format(attempts,msg2) // Empty
{}
Retry attempt:1 for error:Hello
>>> attempts =1
>>> msg2 = "Hello"
>>> print "Retry attempt:{0} for error:{1}".format(attempts,msg2) // With
Values
Retry attempt:1 for error:Hello
>>>
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Converting list of tuple to list
Hello Team, I have a list of tuple say [(1, 2, 1412734464L, 280), (2, 5, 1582956032L, 351), (3, 4, 969216L, 425)] . I need to convert the above as ['1,2,1412734464:280', '2,5,1582956032:351', '3,4,969216:425'] Here is my Solution , Any suggestion or optimizations are welcome . Solution 1: >>> list_tuple = [(1, 2, 1412734464L, 280), (2, 5, 1582956032L, 351), (3, 4, 969216L, 425)] >>> expected_list = [] >>> for elements in list_tuple: ... element = "%s,%s,%s:%s" % (elements) ... expected_list.append(element) ... >>> expected_list ['1,2,1412734464:280', '2,5,1582956032:351', '3,4,969216:425'] Solution 2: >>> list_tuple = [(1, 2, 1412734464L, 280), (2, 5, 1582956032L, 351), (3, 4, 969216L, 425)] >>> expected_list = [] >>> for i in range(len(list_tuple)): ... element = list_tuple[i] ... ex_element = "%s,%s,%s:%s" % (element[0], element[1], element[2], element[3]) ... expected_list.append(ex_element) ... >>> expected_list ['1,2,1412734464:280', '2,5,1582956032:351', '3,4,969216:425'] I know I should have not used len(range()) in Solution 2, any more error please let me know , I am a Linux user on Python 2.7 Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Print Failure or success based on the value of the standalone tool
I have to test a standalone tool from a python script and I am using
os.system() to run the tool . I need to take decision based on the return
value of the standalone tool .
But since os.system merely throws the output value to STDOUT & returns the
exit status (0 => no error) of the shell , how can I make this print Fail
or Pass based on the return value of the standalone tool.
In the below example ( return_values() function)
Example:
# cat standalone_tool.py
#!/usr/bin/env python
# script name: standalone_tool.py
import random
def return_values():
""" My Demo function"""
# After Execution the actual tools returns True, False, None as return
types
x = random.choice([True, False, None])
print x
return x
return_values()
>>> ret = os.system('python standalone_tool.py')
True
>>> ret
0
# vi test_standalone_tool.py
#!/usr/bin/env python
import os
# script name: test_standalone_tool.py
for i in range(2):
retcode = os.system('python standalone_tool.py')
print retcode
if retcode:
print "The standalone tool returned False. Failure"
else:
print "The standalone tool returned True. Successful"
1# python test_standalone_tool.py
None
0
The standalone tool returned True. Successful
True
0
The standalone tool returned True. Successful
The above problem is because we are referring to the exit status of the
shell, Is there an easy way to print Failure or success based on the value
returned by python standalone_tool.py . I am on Linux with Python 2.7 and
sorry for the longish post
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Print Failure or success based on the value of the standalone tool
On Thu, May 10, 2018, 22:31 Rob Gaddi > > > By not using os.system, it's been superseded for reasons exactly like > yours. https://docs.python.org/3/library/subprocess.html is your friend. > Can someone please help me understand this better for me with a program . Will the returncode of subprocess still not be 0 or -ve number ? My requirement is let the test_standalone_tool.py script which is a wrapper around standalone_tool.py print pass /fail based on the possible values I.e True , False and None I'm not sure weather if I need to i.e replace os.system with subprocess or make changes in standalone_tool.py to return 0 or -1( use sys.exit()) Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Regex to extract multiple fields in the same line
Hi Team, I wanted to parse a file and extract few feilds that are present after "=" in a text file . Example , form the below line I need to extract the values present after --struct =, --loc=, --size= and --log_file= Sample input line = '06/12/2018 11:13:23 AM python toolname.py --struct=data_block --log_file=/var/1000111/test18.log --addr=None --loc=0 --mirror=10 --path=/tmp/data_block.txt size=8' Expected output data_block /var/1000111/test18.log 0 8 Here is my sample code , its still not complete , I wanted to use regex and find and extract all the fields after " =", any suggestion or alternative way to optimize this further import re line = '06/12/2018 11:13:23 AM python toolname.py --struct=data_block --log_file=/var/1000111/test18.log --addr=None --loc=0 --mirror=10 --path=/tmp/data_block.txt size=8' r_loc = r"--loc=(\d+)" r_struct = r'--struct=(\w+)' if re.findall(r_loc, line): print re.findall(r_loc, line) if re.findall(r_struct, line): print re.findall(r_struct, line) root@X1:/# python regex_02.py ['0'] ['data_block'] I am a Linux user with python 2.7 Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Re: Regex to extract multiple fields in the same line
On Wed, Jun 13, 2018 at 5:59 PM, Rhodri James wrote:
> On 13/06/18 09:08, Ganesh Pal wrote:
>
>> Hi Team,
>>
>> I wanted to parse a file and extract few feilds that are present after "="
>> in a text file .
>>
>>
>> Example , form the below line I need to extract the values present after
>> --struct =, --loc=, --size= and --log_file=
>>
>> Sample input
>>
>> line = '06/12/2018 11:13:23 AM python toolname.py --struct=data_block
>> --log_file=/var/1000111/test18.log --addr=None --loc=0 --mirror=10
>> --path=/tmp/data_block.txt size=8'
>>
>
> Did you mean "--size=8" at the end? That's what your explanation implied.
Yes James you got it right , I meant "--size=8 " .,
Hi Team,
I played further with python's re.findall() and I am able to extract all
the required fields , I have 2 further questions too , please suggest
Question 1:
Please let me know the mistakes in the below code and suggest if it can
be optimized further with better regex
# This code has to extract various the fields from a single line (
assuming the line is matched here ) of a log file that contains various
values (and then store the extracted values in a dictionary )
import re
line = '06/12/2018 11:13:23 AM python toolname.py --struct=data_block
--log_file=/var/1000111/test18.log --addr=None --loc=0 --mirror=10
--path=/tmp/data_block.txt --size=8'
#loc is an number
r_loc = r"--loc=([0-9]+)"
r_size = r'--size=([0-9]+)'
r_struct = r'--struct=([A-Za-z_]+)'
r_log_file = r'--log_file=([A-Za-z0-9_/.]+)'
if re.findall(r_loc, line):
print re.findall(r_loc, line)
if re.findall(r_size, line):
print re.findall(r_size, line)
if re.findall(r_struct, line):
print re.findall(r_struct, line)
if re.findall(r_log_file, line):
print re.findall(r_log_file, line)
o/p:
root@X1:/Play_ground/SPECIAL_TYPES/REGEX# python regex_002.py
['0']
['8']
['data_block']
['/var/1000111/test18.log']
Question 2:
I tried to see if I can use re.search with look behind assertion , it
seems to work , any comments or suggestions
Example:
import re
line = '06/12/2018 11:13:23 AM python toolname.py --struct=data_block
--log_file=/var/1000111/test18.log --addr=None --loc=0 --mirror=10
--path=/tmp/data_block.txt --size=8'
match = re.search(r'(?P(?<=--loc=)([0-9]+))', line)
if match:
print match.group('loc')
o/p: root@X1:/Play_ground/SPECIAL_TYPES/REGEX# python regex_002.py
0
I want to build the sub patterns and use match.group() to get the values
, some thing as show below but it doesn't seem to work
match = re.search(r'(?P(?<=--loc=)([0-9]+))'
r'(?P(?<=--size=)([0-9]+))', line)
if match:
print match.group('loc')
print match.group('size')
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Regex to extract multiple fields in the same line
>
{'struct': 'data_block', 'log_file': '/var/1000111/test18.log', 'loc':
> '0', 'size': '8'}
>
>
MARB, as usual the solution you you look nice, Thanks for the excellent
solutions
>>> regex = re.compile (r"--(struct|loc|size|mirror|l
og_file)\s*=\s*([^\s]+)")
>>> regex.findall (line)
[('struct', 'data_block'), ('log_file', '/var/1000111/test18.log'), ('loc',
'0'), ('mirror', '10')]
Frederic , this look great, thanks for your response
--
https://mail.python.org/mailman/listinfo/python-list
For specific keys , extract non empty values in a dictionary
*How do I check few specific/selected keys in a dictionary and extract
their values if they are not empty*
*Example : Extract the values for key "one","three","seven" and "nine” if
they are not empty*
*Input :*
*o_num = {'one': 1,*
* 'three': 3,*
* 'bar': None,*
* 'five' : 5,*
* 'rum' : None,*
* 'seven' : None,*
* 'brandy': None,*
* 'nine' : 9,*
* 'gin': None}*
*Output:*
*1 3 9*
Here is my solution , Please review the below code and let me know your
suggestion .
#/usr/bin/python
o_num = {'one': 1,
'three': 3,
'bar': None,
'five' : 5,
'rum' : None,
'seven' : None,
'brandy': None,
'nine' : 9,
'gin': None}
args_list = ["one","three","seven","nine"]
args_dict = dict( (k,v) for k, v in o_num.items() if v and k in args_list )
print args_dict
o/p:
root@X1:/Play_ground/DICT# python hello.py
{'nine': 9, 'three': 3, 'one': 1}
Also, Is unpacking the elements in the o_num dictionary as shown below
fine or are there any other alternatives
arg1, arg2, arg3, arg4, = map(args_dict.get, ('one', 'three', 'seven',
'nine'))
print arg1, arg2, arg3, arg4
I am a Python 2.7 user and on Linux box
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: For specific keys , extract non empty values in a dictionary
> >>> {k: o_num[k] for k in wanted & o_num.keys() if o_num[k] is not None}
Thanks peter this looks better , except that I will need to use the
logial 'and' operator or else I will get a TypeError
>>> {k: o_num[k] for k in wanted & o_num.keys() if o_num[k] is not None}
TypeError: unsupported operand type(s) for &: 'set' and 'list'
Example :
>>>print {k: o_num[k] for k in wanted and o_num.keys() if o_num[k] is not
None}
{'nine': 9, 'five': 5, 'three': 3, 'one': 1}
On Sat, Jun 16, 2018 at 11:12 PM, Peter Otten <__pete...@web.de> wrote:
> Ganesh Pal wrote:
>
> > *How do I check few specific/selected keys in a dictionary and extract
> > their values if they are not empty*
>
> You mean not None.
>
> > o_num = {'one': 1,
> > 'three': 3,
> > 'bar': None,
> > 'five' : 5,
> > 'rum' : None,
> > 'seven' : None,
> > 'brandy': None,
> > 'nine' : 9,
> > 'gin': None}
>
> > args_list = ["one","three","seven","nine"]
>
> > *Output:*
> >
> > *1 3 9*
>
> >>> wanted = {"one", "three", "seven", "nine"}
> >>> {k: o_num[k] for k in wanted & o_num.keys() if o_num[k] is not None}
> {'one': 1, 'nine': 9, 'three': 3}
>
> > I am a Python 2.7 user and on Linux box
>
> You have to replace keys() with viewkeys() in Python 2.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
write the values of an ordered dictionary into a file
Hi Team
I need to write the values of an ordered dictionary into a file . All
values should be in a single row with a header list
*Example:*
*student = [("NAME", "John"),*
* ("AGE", 28),*
* ("SCORE", 13),*
* ("YEAR", 2018),*
* ("FEE", 250)]*
*student = OrderedDict(student)*
*The OrderedDict Should be append at the end of the file as as shown below.*
*# tail -2 /tmp/student_record.txt *
*.*
*||STUDENT NAME||STUDENT AGE||MARKS SCORED||PASSED YEAR||FEES PAID||*
*||John|| 28 || 13|| 2018 || 250 ||*
Questions:
(1) Below is my partial solution , any comments and suggestions ( I am not
to get the “||” delimiter correctly, trying it )
#!/usr/bin/python
# A Python program to write the values of an OderedDict into a file
# The values should be formatted correctly under its headers
from collections import OrderedDict
tmp = '/tmp/student_record.txt'
student = [("NAME", "John"),
("AGE", 28),
("SCORE", 13),
("YEAR", 2018),
("FEE", 250)]
student = OrderedDict(student)
header_list = ["STUDENT NAME", "STUDENT AGE", "MARKS SCORED", "PASSED YEAR",
"FEES PAID"]
header_string = '||' + '||'.join(header_list) + '||'
with open(tmp, 'a') as fd:
for item in header_string:
fd.write("%s" % (item))
for value in student.values():
fd.write("\n")
fd.write("||")
fd.write("%s" % (value))
*output:*
*root@X1:/Play_ground/SPECIAL_TYPES# cat /tmp/student_record.txt*
*||STUDENT NAME||STUDENT AGE||MARKS SCORED||PASSED YEAR||FEES PAID||*
*||John*
*||28*
*||13*
*||2018*
(2)Any alternative way to solve this easily and store the data in the
requested format (can I still use cvs writer to format this)
I am a Linux user with Python 2.7
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
try except inside a with open
Dear python Friends,
I need a quick suggestion on the below code.
def modify_various_line(f):
""" Try modifiying various line """
try:
f.write('0123456789abcdef')
f.seek(5) # Go to the 6th byte in the file
print f.read(1)
f.seek(-3, 2) # Go to the 3rd byte before the end
print f.read(1)
f.write('END')
except IOError as e:
raise
return True
def h():
try:
with open('/tmp/file.txt', 'r+') as f:
try:
modify_various_line(f)
except Exception as e:
print e
except IOError as e:
print(e)
h()
gpal-cwerzvd-1# python ganesh1.py
5
d
gpal-cwerzvd-1# cat /tmp/file.txt
0123456789abcdefEND#
(1) Can we use try and expect in a 'with open' function as shown in the
below example code .
(2) If I hit any other exceptions say Value-error can I catch them as
show below
try:
with open('/tmp/file.txt', 'r+') as f:
try:
modify_various_line(f)
value_list = [str(value) for value in somedict.values()]]
except (ValueError,IOError) as e:
print e
except IOError as e:
PS: I don't want to use other way of opening file say file =
open(“testfile.txt”,”w”) and also want to retain modify_various_line(f)
function ,
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: try except inside a with open
>
>
>
> (1) Since this function always returns True (if it returns at all), what
> is the point? There's no point checking the return result, since it's
> always true, so why bother returning anything?
>
>
If I don't return anything from a function it returns None. But would it
be better if for the function i.e modify_various_line(f) to atleast catch
or handle exceptions and log it for debugging purpose
Example:
def modify_various_line(f):
""" Try modifiying various line """
try:
f.write('0123456789abcdef')
f.seek(5) # Go to the 6th byte in the file
print f.read(1)
f.seek(-3, 2) # Go to the 3rd byte before the end
print f.read(1)
f.write('END')
except IOError as e:
logging.error("Error: got %s" , % (str(e)))
--
https://mail.python.org/mailman/listinfo/python-list
Better way / regex to extract values form a dictionary
I have one of the dictionary values in the below format
'/usr/local/ABCD/EDF/ASASAS/GTH/HELLO/MELLO/test04_Failures.log'
'/usr/local/ABCD/EDF/GTH/HEL/OOLO/MELLO/test02_Failures.log'
'/usr/local/ABCD/EDF/GTH/BEL/LO/MELLO/test03_Failures.log'
I need to extract the file name in the path example, say test04_Failure.log
and testcase no i.e test04
Here is my solutions:
gpal-cwerzvd-1# vi filename.py
import re
Common_dict = {}
Common_dict['filename'] =
'/usr/local/ABCD/EDF/GTH/HELLO/MELLO/test04_Failures.log'
def return_filename_test_case(filepath):
if filepath:
filename = re.findall(r'(test\d{1,4}_\S+)', filepath)
if filename:
testcase = re.findall(r'(test\d{1,4})', ''.join(filename))
return filename, testcase
if Common_dict['filename']:
path = Common_dict['filename']
fname, testcase = return_filename_test_case(path)
print fname, testcase
op:
qerzvd-1# python filename.py
['test04_Failures.log']
['test04']
Please suggest how can this code can be optimized further looks messy ,
what would be your one liner or a simple solution to return both test-case
no and filename
I am on Python 2.7 and Linux
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Better way / regex to extract values form a dictionary
> The dictionary is irrelevant to your question. It doesn't matter whether
> the path came from a dict, a list, read directly from stdin, an
> environment variable, extracted from a CSV file, or plucked directly from
> outer space by the programmer. The process remains the same regardless of
> where the path came from.
>
Thanks I was ignorant about this , your solution looks cool . One last
question , does it makes sense to check if the values are not none ( If I
plan to use the value for next computation )
if not all(filename, testcase):
raise ValueError("Error getting filename or testcase no")
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
how to optimize the below code with a helper function
Hi Team, Iam on python 2.7.10 and Linux. I have a python function where the similar kind of pattern repeating 100 of times Sample code snippet: test01_log = os.path.join(LOG_DIR, "test01.log") cls.get_baddr['test01'] = failure.run_tool( test01_log, object="inode", offset="18", size="4", optype="set") test02_log = os.path.join(LOG_DIR, "test02.log") cls.get_baddr['test02'] = failure.run_tool( test02_log, lin=lin_02, object="lin", offset="100", size="5", optype="set") .. test100_log = os.path.join(LOG_DIR, "test100.log") cls.get_baddr['test100'] = failure.run_tool( test02_log, baddr=lin_02, object="baddr", offset="100", size="5", optype="set") (1) Any tips how I can optimize this i.e test case, should have a helper function that all test cases call. (2) Also note that failure.run_tool function can have variable number of argments how to handle this in the helper function? Thanks in advance Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
sys.exit(1) vs raise SystemExit vs raise
I m on python 2.7 and Linux , I have a simple code need suggestion if I
I could replace sys.exit(1) with raise SystemExit .
==Actual code==
def main():
try:
create_logdir()
create_dataset()
unittest.main()
except Exception as e:
logging.exception(e)
sys.exit(EXIT_STATUS_ERROR)
if __name__ == '__main__':
main()
==Changed Code==
def main():
try:
create_logdir()
create_dataset()
unittest.main()
except Exception as e:
logging.exception(e)
raise SystemExit
if __name__ == '__main__':
main()
2. All the functions in try block have exception bubbled out using raise
Example for create_logdir() here is the function definition
def create_logdir():
try:
os.makedirs(LOG_DIR)
except OSError as e:
sys.stderr.write("Failed to create log directory...Exiting !!!")
raise
print "log file: " + corrupt_log
return True
def main():
try:
create_logdir()
except Exception as e:
logging.exception(e)
raise SystemExit
(a) In case if create_logdir() fails we will get the below error ,is
this fine or do I need to improve this code.
Failed to create log directory...Exiting !!!ERROR:root:[Errno 17] File
exists: '/var/log/dummy'
Traceback (most recent call last):
File "corrupt_test.py", line 245, in main
create_logdir()
File "corrupt_test.py", line 53, in create_logdir
os.makedirs(LOG_DIR)
File "/usr/local/lib/python2.7/os.py", line 157, in makedirs
OSError: [Errno 17] File exists: '/var/log/dummy'
3. Can I have just raise , instead of SystemExit or sys.exit(1) . This
looks wrong to me
def main():
try:
create_logdir()
except Exception as e
logging.exception(e)
raise
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: sys.exit(1) vs raise SystemExit vs raise
> > > No; raise SystemExit is equivalent to sys.exit(0); you would need raise > SystemExit(1) to return 1. > Thanks will replace SystemExit with SystemExit(1) . > Why do you want to do this, though? What do you think you gain from it? > Iam trying to have a single exit point for many functions: example create_logdir() , create_dataset() and unittest.main() will bubble out an exception using raise I would want to terminate the program when this happens . Do you see any problem if *raise *SystemExit(1) is used in the except block ? *def *main(): *try*: create_logdir() create_dataset() unittest.main() *except *Exception *as *e: logging.exception(e) *raise *SystemExit(1) I see the below error only on pdb so thinking whats wrong in the above code ? “*Exception AttributeError: "'NoneType' object has no attribute 'path'" in ignored “ * (Pdb) n SystemExit: SystemExit() > /var/crash/local_qa/bin/corrupt_test.py(253)() -> main() (Pdb) n --Return-- > /var/crash/local_qa/bin/corrupt_test.py(253)()->None -> main() (Pdb) n Exception AttributeError: "'NoneType' object has no attribute 'path'" in ignored -- https://mail.python.org/mailman/listinfo/python-list
Skipping test using unittest SkipTest and exit status
Hi Team,
Iam on python 2.7 and Linux . I need inputs on the below program ,
Iam skipping the unittest from setUpClass in following way # raise
unittest.SkipTest(message)
The test are getting skipped but I have two problem .
(1) This script is in turn read by other scripts which considers the
test have passed based on the scripts return code , but the test have
actually been skipped , How do include an exit status to indicates that
the test have failed
(2) Why is the message in the raise statement i.e raise
unittest.SkipTest("Class setup failed skipping test") not getting
displayed .
Also thinking if we could replace raise unittest.SkipTest with assert
statement ?
Sample code:
#!/usr/bin/env python
import unittest
import logging
class ScanTest(unittest.TestCase):
@classmethod
def setUpClass(self):
"""
Initial setup before unittest run
"""
pdb.set_trace()
self.scan = False
if not self.scan:
logging.error("Failed scanning ")
raise unittest.SkipTest("Class setup failed skipping test")
self.data = True
if not self.data:
logging.error("Failed getting data ")
raise unittest.SkipTest("Class setup failed skipping test")
logging.info("SETUP.Done")
def test_01_inode_scanion(self):
""" test01: inode scanion """
logging.info("### Executing test01: ###")
@classmethod
def tearDownClass(self):
""" Cleanup all the data & logs """
logging.info("Cleaning all data")
def main():
""" ---MAIN--- """
try:
unittest.main()
except Exception as e:
logging.exception(e)
sys.exit(1)
if __name__ == '__main__':
main()
Sample output
gpal-ae9703e-1# python unitest1.py
ERROR:root:Failed scanning
s
--
Ran 0 tests in 0.000s
OK (skipped=1)
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Skipping test using unittest SkipTest and exit status
>
> > Hi Team,
> >
> > Iam on python 2.7 and Linux . I need inputs on the below program ,
>
> "I am" is two words, not one. I hope you wouldn't write "Youare"
> or "Heis" :-) Whenever you write "Iam", I read it as the name "Ian", which
> is very distracting.
>
>
I am lazy fellow and you are smart guy. just a sentence with few words .
Take care :)
> > Iam skipping the unittest from setUpClass in following way # raise
> > unittest.SkipTest(message)
> >
> > The test are getting skipped but I have two problem .
> >
> > (1) This script is in turn read by other scripts which considers the
> > test have passed based on the scripts return code , but the test have
> > actually been skipped , How do include an exit status to indicates
> that
> > the test have failed
>
> But the test *hasn't* failed. A skipped test is not a failed test.
>
> If you want the test to count as failed, you must let it fail. You can use
> the fail() method for that.
>
> https://docs.python.org/2/library/unittest.html#unittest.TestCase.fail
>
>
1. How about raising failureException :
I was thinking of using failureException instead of fail() method , If I
replace my code with raise unittest.TestCase.failureException("class setup
failed")
The script show the below output , this looks fine for me. Do you see any
problems with this ?
gpal-ae9703e-1# python unitest1.py
ERROR:root:Failed scanning
E
==
ERROR: setUpClass (__main__.ScanTest)
--
Traceback (most recent call last):
File "unitest1.py", line 20, in setUpClass
raise unittest.TestCase.failureException("class setup failed")
AssertionError: class setup failed
--
Ran 0 tests in 0.000s
FAILED (errors=1)
2. I find assert and raise RunTimeError also fitting my program ,please
suggest whats best form unittest fixture point of view.
if not self.scan:
logging.error("Failed scanning ")
assert False, "Class setup failed skipping test"
if not self.scan:
logging.error("Failed scanning ")
raise RuntimeError.
My overall ,idea is Setup class fails then don't run any of the next
statements and exit the tests.
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
re.search - Pattern matching review
Dear Python friends,
I am on Python 2.7 and Linux . I am trying to extract the address
"1,5,147456:8192" from the below stdout using re.search
(Pdb) stdout
'linux-host-machine-1: Block Address for 1,5,27320320:8192 (block
1,5,147456:8192) --\nlinux-host-machine-1: magic
0xdeaff2fe mark_cookie 0x300a\n'
(Pdb) type(stdout)
Here is the code I have come up with, this looks buggy please review
the same and suggest any better ways to code the same
the same may be splitlines() , re.complie() etc , my intention is to
just match 1,5,147456:8192 and return the same.
#Sample code
import re
import subprocess_run
def get_block():
try:
cmd = "get_block_info -l"
# stdout is the output retrieved by subprocess.Popen()
stdout, stderr, exitcode = subprocess_run(cmd)
search_pat = 'Block Address.* \(block (\d+),(\d+),(\d+):(\d+)'
matched = re.search(search_pat, stdout)
block = (int(matched.group(1)),
int(matched.group(2)),
int(matched.group(3)),
int(matched.group(4)),
)
except IOError, e:
logging.warning('Error reading lines from "%s" (%s).'
% (cmd, e))
if block is None:
logging.error("block not found")
return False
logging.info("block not found")
return block
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
re.search - Pattern matching review ( Apologies re sending)
Dear Python friends,
I am on Python 2.7 and Linux . I am trying to extract the address
"1,5,147456:8192" from the below stdout using re.search
(Pdb) stdout
'linux-host-machine-1: Block Address for 1,5,27320320:8192 (block
1,5,147456:8192) --\nlinux-host-machine-1: magic
0xdeaff2fe mark_cookie 0x300a\n'
(Pdb) type(stdout)
Here is the code I have come up with, this looks buggy please review
the same and suggest any better ways to code.
Could we use splitlines() or re.complie() etc , my intention is to
match 1,5,147456:8192 and return the same.
#Sample code
import re
import subprocess_run
def get_block():
try:
cmd = "get_block_info -l"
# stdout is the output retrieved by subprocess.Popen()
stdout, stderr, exitcode = subprocess_run(cmd)
search_pat = 'Block Address.* \(block (\d+),(\d+),(\d+):(\d+)'
matched = re.search(search_pat, stdout)
block = (int(matched.group(1)),
int(matched.group(2)),
int(matched.group(3)),
int(matched.group(4)),
)
except IOError, e:
logging.warning('Error reading lines from "%s" (%s).'
% (cmd, e))
if block is None:
logging.error("block not found")
return False
logging.info("block not found")
return block
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: re.search - Pattern matching review ( Apologies re sending)
> Perhaps: > map(int, re.search(search_pat, stdout).groups()) > > > Thanks Albert map saved me many lines of code but map returns a list I will have to convert the list to string again Below is how Iam planning to teh conversion >>> block = map(int, re.search(search_pat, stdout).groups()) >>> print block ['1,2:122'] >>> s1 = ','.join(str(n) for n in block) >>> print s1 1,2:122 >>> str(s1) '1,2:122' Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Re: re.search - Pattern matching review ( Apologies re sending)
The matched.groups() will group the pattern based with ","
(Pdb) matched.groups()
*('1', '0', '1375772672', '8192')*
but I wanted to retain the output as *'1,0,1375772672:8192' ,*
(Pdb) matched.groups()
('1', '0', '1375772672', '8192')
(Pdb) matched.group()
'Block Address for 1,0,1376034816:8192 (block *1,0,1375772672:8192*'
Regards,
Ganesh
On Sun, May 29, 2016 at 11:53 AM, Ganesh Pal wrote:
>
>
>
>> Perhaps:
>> map(int, re.search(search_pat, stdout).groups())
>>
>>
>>
> Thanks Albert map saved me many lines of code but map returns a list I
> will have to convert the list to string again
> Below is how Iam planning to teh conversion
> >>> block = map(int, re.search(search_pat, stdout).groups())
> >>> print block
> ['1,2:122']
> >>> s1 = ','.join(str(n) for n in block)
> >>> print s1
> 1,2:122
> >>> str(s1)
> '1,2:122'
>
> Regards,
> Ganesh
>
--
https://mail.python.org/mailman/listinfo/python-list
python parsing suggestion
Hi ,
Trying to extract the '1,1,114688:8192' pattern form the below output.
pdb>stdout:
'3aae5d0-1: Parent Block for 1,1,19169280:8192 (block 1,1,114688:8192)
--\n3aae5d0-1:
magic 0xdeaff2fe mark_cookie
0x\ngpal-3aae5d0-1: super.status
3super.cookie 390781895\ngpal-3aae5d0-1:
cg_xth 0
I am on python 2.7 and Linux the below code sample is working fine (
please raise the error if u find it will help me improve this
codebetter)
def check_block(block):
"""
Trying to extract the '1,1,114688:8192' pattern from the above output.
"""
logging.info('Determining history block for block %s' % (block))
parent_block = None
node_id = block.split(",")[0]
cmd = ("get_block_info -l" % (node_id, block))
logging.info(cmd)
stdout, stderr, exitcode = run(cmd)
try:
parent_block = stdout.strip().split('\n')[0].split()[6][:-1]
except (IndexError, ValueError):
logging.error('Error determining history block for %s.' % (block))
return False
if re.search(r'(\d+),(\d+),(\d+):(\d+)', parent_block):
logging.info('Found history block %s for data block %s' %
(parent_block, block))
return parent_block
return False
Need suggestion for the below 3 points:
1. Is parsing with stdout.strip().split('\n')[0].split()[6][:-1]
sufficient do I need to add extra check ? it looks fine for me though.
2. Better ways to achieve the same output we need to parse is a string
3. Is re.search(r'(\d+),(\d+),(\d+):(\d+)', parent_block) needed ? I
added as an extra check ,any ideas on the same
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: re.search - Pattern matching review
On Sun, May 29, 2016 at 10:32 PM, Matt Wheeler wrote:
>
>
> This doesn't seem to exactly match your code below, i.e. your code is
> attempting to construct a tuple from groups 1 through 4. To meet this
> specification I could just `return re.search('(?<=\(block
> )[^(]*(?=\))', stdout).group()`
>
> Thanks Matt for the reply and lovely analysis . I was trying to
complicate the simple task :(
Here is how the code looks now , the whole idea was just to match the
pattern and return it
def get_block(block):
cmd = "get_block_info -l"
stdout, stderr, exitcode = subprocess_run(cmd)
#Grab the block from the stdout
block = re.search('(?<=\(block )[^(]*(?=\))', stdout).group()
# check the pattern
matched = re.search(r'(\d+),(\d+),(\d+):(\d+)', block)
if matched:
logging.info('block found")
return block
else:
logging.info('block not found")
I had one final question. I was thinking if we included a try -expect
block to catch the failures of re.search as shown below.
what kind of specific exception can we add ( just the AttributeError
Exception or any thing else )
Example :
try:
block = re.search('(?<=\(block )[^(]*(?=\))', stdout).group()
matched = re.search(r'(\d+),(\d+),(\d+):(\d+)', block)
except AttributeError
logging.error(' Error: while determining the block ")
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: re.search - Pattern matching review
Thanks works fine : ) -- https://mail.python.org/mailman/listinfo/python-list
one command on backslash and space for review
Hello Team, I am on python 2.7 and Linux , I want to form the below sample command so that I could run it on the shell. Command is --> run_parallel -za1 -s 'daemon -cf xyz; sleep 1' Here is how I formed the command and it seems to look fine and work fine , but I think it could still be better any idea ? >>> cmd = "run_parallel -za" + str(number) + \ ... " -s" + " \'daemon -cf xyz; sleep 1\'" >>> cmd "run_parallel -za1 -s 'daemon -cf xyz; sleep 1'" >>> Looking for thoughts around: 1. If backslash are syntactically correct 2. " -s" , starting with a leading space , may be its not a good idea , other ways 3. Iam running sleep command on the cluster i.e , how could I make it look Python or its fine to have sleep ? Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Re: one command on backslash and space for review
On Jul 10, 2016 11:14 PM, "Ian Kelly" wrote: > They're correct, but using them before single quotes in a string > delimited by double quotes is unnecessary. Thanks . > > 3. Iam running sleep command on the cluster i.e , how could I make it > > look Python or its fine to have sleep ? > > I don't understand the question. > Sorry I was in sleep 😂😂 I typed something wrong . Let me explain it. Better In the below command I.e 'run_parallel -za1 -s 'daemon -cf xyz; sleep 1' We have a sleep 1 that's run run as part of abovry shell command . It looks ok but is there a way to use something alternative to sleep was my question. I guess the answer is " no" , because we are executing shell sleep command I think something pythonic like time.sleep is not possible. -- https://mail.python.org/mailman/listinfo/python-list
Re: one command on backslash and space for review
>
>
>
> >>>> cmd = "run_parallel -za" + str(number) + \
> > ... " -s" + " \'daemon -cf xyz; sleep 1\'"
>
> cmd = "run_parallel -za{} -s 'daemon -cf xyz; sleep 1'".format(number)
>
>
How will I format number to strings using .format ??
Example
>>> str(num)
'100'
>>> cmd = "run_parallel -za{} -s 'daemon -cf xyz; sleep 1'".format(str(num))
>>> cmd
"run_parallel -za100 -s 'daemon -cf xyz; sleep 1'“.
Will Something like format(str(num)) work ? it working though
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
use import *
Hi Team , I am a Linux user on python 2,6 . I have a very simple question I was going the zen of python by Tim peters and found an example that demonstrates Explicit is better than implicit """Load the cat, dog, and mouse models so we can edit instances of them.""" def load(): from menagerie.cat.models import * from menagerie.dog.models import * from menagerie.mouse.models import * #--- def load(): from menagerie.models import cat as cat_models from menagerie.models import dog as dog_models from menagerie.models import mouse as mouse_models #--- print 'Explicit is better than implicit.' I had a question on the above example 1. I haven't used " from menagerie.cat.models import * is it a good programming practice to use import * ? if answer is "NO " then are there situation where you are forced to use import * Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Intitalize values for a class
Hello team,
I am a python 2.7 user on Linux. I will need feedback on the below program
as I'm new to oops .
#!/usr/bin/python
class System(object):
'''Doc - Inside Class '''
def __init__(self, params=None):
if params is None:
self.params = {'id': '1',
'name': 's-1'}
print self.params
if type(params) is dict and params.get('id') == '0':
raise ValueError('ERROR: id 0 is reserved !! ')
#print self.params
else:
self.params = params
print self.params
# Test all conditions
#case 0 - Default should create {'id': '1','name': 's-1'}
#s0 = System()
#Case 1 (id has value '0')
#test1_params = {'id': '0', 'name': 's-0'}
#s1 = System(params=test1_params)
#Case 2 (id has some other values)
#test2_params = {'id': '10', 'name': 's-10'}
#s2 = System(params=test2_params)
Question:
I have to initialize the values the below class such that
1. Intitalize default values if nothing is supplied by the username i.e
self.params = {'id': '1', 'name': 's-1'}
2. I need to raise an Exception if the value for the key params[id] is '0'.
3. It should work if params[I'd] has values other than (1) and (2)
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: initialize the values of the class
Sorry for reposting, typo in the subject line !
On Fri, Nov 23, 2018, 19:11 Ganesh Pal Hello team,
>
> I am a python 2.7 user on Linux. I will need feedback on the below program
> as I'm new to oops .
>
> #!/usr/bin/python
>
>
> class System(object):
>
> '''Doc - Inside Class '''
>
> def __init__(self, params=None):
>
>if params is None:
>
> self.params = {'id': '1',
>
> 'name': 's-1'}
>
> print self.params
>
>if type(params) is dict and params.get('id') == '0':
>
> raise ValueError('ERROR: id 0 is reserved !! ')
>
>#print self.params
>
>else:
>
> self.params = params
>
> print self.params
>
> # Test all conditions
>
> #case 0 - Default should create {'id': '1','name': 's-1'}
> #s0 = System()
>
> #Case 1 (id has value '0')
> #test1_params = {'id': '0', 'name': 's-0'}
> #s1 = System(params=test1_params)
>
>
> #Case 2 (id has some other values)
> #test2_params = {'id': '10', 'name': 's-10'}
> #s2 = System(params=test2_params)
>
>
> Question:
>
> I have to initialize the values the below class such that
>
> 1. Intitalize default values if nothing is supplied by the username
> i.e self.params = {'id': '1', 'name': 's-1'}
>
> 2. I need to raise an Exception if the value for the key params[id] is '0'.
>
> 3. It should work if params[I'd] has values other than (1) and (2)
>
> Regards,
> Ganesh
>
>
>
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Intitalize values for a class
On Fri, Nov 23, 2018, 19:30 Bob Gailer What kind of feedback do you want? > Wanted to know if there is any problem in the code and if you can review it :-) > -- https://mail.python.org/mailman/listinfo/python-list
How to remove "" from starting of a string if provided by the user
How to remove " from the starting and ending of a string , before
comparison . Here is an example and my solution wtih eval ( I am advised
not to use this one) , please suggest an alternative . I am on linux and
python 2.7
g1@X1:/tmp$ cat file2.py
#!/usr/bin/python
# Case 1 - server2 file is "'/fileno_100.txt'"
stat={}
stat['server1'] = '/fileno_100.txt'
stat['server2'] = "'/fileno_100.txt'"
if stat['server1'] == eval(stat['server2']):
print "OK"
# Case 2 - server2 file is '/fileno_100.txt'
stat['server2'] = "'/fileno_100.txt'"
if stat['server1'] == eval(stat['server2']):
print "OK"
# Case 3 - server2 file can be in (a) '/fileno_100.txt' or (b) :
"'/fileno_100.txt'" format
g1@X1:/tmp$ python file2.py
OK
OK
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to remove "" from starting of a string if provided by the user
The possible value of stat['server2'] can be either (a)
"'/fileno_100.txt'" or (b) '/fileno_100.txt' .
How do I check if it the value was (a) i.e string started and ended
with a quote , so that I can use ast.literal_eval()
>>> import ast
>>> stat = {}
>>> stat['server2'] = "'/fileno_100.txt'"
>>> stat['server2'] = ast.literal_eval(stat['server2'])
>>> print stat['server2']
/fileno_100.txt
>>>
>>> if stat['server2'].startswith("\"") and stat['server2'].endswith("\""):
...stat['server2'] = ast.literal_eval(stat['server2'])
...
>>>
I tried startswith() and endswith(), doesn't seem to work ?. Is there
a simpler way ?
Regards,
Ganesh
On Tue, Aug 11, 2020 at 4:06 AM MRAB wrote:
> On 2020-08-10 19:35, Ganesh Pal wrote:
> > How to remove " from the starting and ending of a string , before
> > comparison . Here is an example and my solution wtih eval ( I am advised
> > not to use this one) , please suggest an alternative . I am on linux and
> > python 2.7
> >
> > g1@X1:/tmp$ cat file2.py
> > #!/usr/bin/python
> >
> > # Case 1 - server2 file is "'/fileno_100.txt'"
> > stat={}
> > stat['server1'] = '/fileno_100.txt'
> > stat['server2'] = "'/fileno_100.txt'"
> >
> > if stat['server1'] == eval(stat['server2']):
> > print "OK"
> >
> > # Case 2 - server2 file is '/fileno_100.txt'
> > stat['server2'] = "'/fileno_100.txt'"
> >
> > if stat['server1'] == eval(stat['server2']):
> > print "OK"
> >
> >
> > # Case 3 - server2 file can be in (a) '/fileno_100.txt' or (b) :
> > "'/fileno_100.txt'" format
> >
> > g1@X1:/tmp$ python file2.py
> > OK
> > OK
> >
> You could strip off the quotes with the .strip method or use
> literal_eval from the ast module.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
how to make the below code look better
Hello team,
I need suggestion to improve the below code , Iam on Linux and python 2.7
if not os.path.ismount("/tmp"):
sys.exit("/tmp not mounted.")
else:
if create_dataset() and check_permission():
try:
run_full_back_up()
run_partial_back_up()
except Exception, e:
logging.error(e)
sys.exit("Running backup failed")
if not validation_errors():
sys.exit("Validation failed")
else:
try:
compare_results()
except Exception, e:
logging.error(e)
sys.exit("Comparing result failed")
Question 1:
1. if create_dataset() and check_permission():
Iam assuming that if statement will be executed only if both the
functions are true ?
2. Can I have a if statement within if else ? , some how I feel its messy
3. Any other suggestion ? please
--
https://mail.python.org/mailman/listinfo/python-list
Re: how to make the below code look better
On Wed, Dec 2, 2015 at 6:00 PM, Chris Angelico wrote:
> If both the functions return true values, yes. You have an indentation
> error there, but I'm assuming you meant to have the try/except
> indented further.
>
Correct I had meant to have try/except indented further.
>> 2. Can I have a if statement within if else ? , some how I feel its messy
>
> Certainly you can! However, most (maybe all) of your 'if' statements
> are checking for error conditions, so the easiest solution is to
> simply exit right away (either with sys.exit or by raising an
> exception).
Yes agreed , have included the suggestion
>> 3. Any other suggestion ? please
>
> The first suggestion I'd make is to avoid the comma syntax for
> exception handling. Replace "except Exception, e:" with "except
> Exception as e:". That's a trivial change of syntax that shouldn't
> affect your code at all; consider it low-hanging fruit.
>
,Are we avoiding comma syntax because it's a bad convention or will
it have any side effects? In fact I have used this comma syntax in
most of my code . I will clean the code latter as it seems working
fine
> The second thing to look at is the way you're 'handling' those errors.
> All you do is log the error and exit. So you can skip the except
> clauses altogether, and just do this:
>
> if not os.path.ismount("/tmp"):
> sys.exit("/tmp not mounted.")
> if create_dataset() and check_permission():
> run_full_back_up()
> run_partial_back_up()
> if not validation_errors(): # CHECK ME
> sys.exit("Validation failed")
> compare_results()
>
>
Agreed.
> Check the logic of validation_errors, though. If it returns something
> True, does that mean there were errors or there weren't? If it means
> there were errors, then the 'not' seems to me to be wrong; but if it
> means there weren't, then I would rename it "validation_successful"
> rather than "validation_errors".
>
> Also worth checking: If you can't create the data set or the
> permission check fails, what should it do? Should it terminate with an
> error? Currently, it carries on and does the validation, which is
> probably not right. If it ought to terminate straight away, you can
> write the whole program in "fail and bail" style:
>
> if not create_dataset():
> sys.exit("Data set creation failed")
> if not check_permission():
> sys.exit("Permission check failed")
>
Thanks for other suggestions also
--
https://mail.python.org/mailman/listinfo/python-list
storing test logs under /var/log/
Hi Team ,
I would need few tips from your past experiences on how to store
the test logs
My requirement is to capture log under /var/log/ directory every time
the test is run . The test will create one small log files which are
around 1KB in size .
Here is how I plan to approach this , create directory based on
current timesamp and store the logs in it .
Sample code :
time_now = datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S') + "/"
LOG_DIR = "" + time_now
try:
retcode = os.makedirs(LOG_DIR)
if retcode:
raise Exception(
"Faild to create log directory. mkdir %s failed !!!"
% LOG_DIR)
except Exception, e:
sys.exit("Failed to create log directory...Exiting !!!")
1. Do I need to add the cleanup code to remove the log directory
say /var/log/test_log/2015-11-25_04-07-48/ , because we might have
many more directories like this when test are run multiple times , Iam
avoiding because the test will be run
2/3 times max and file sizes are also very small
2. Any better suggestion for my use case.
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: storing test logs under /var/log/
> Finally. sys.exit accepts an integer, not a string.
>
Most of code uses sys.exit("some error message") , I did notice
that the error message is not displayed by sys .exit("some error
message") , do u mean that using string is not advisable with
sys.exit ?
How to I display error messages with sys.exit then ?
PS:Thanks for all your previous comments , all were quite helpful .
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: storing test logs under /var/log/
> Wrong question; if you want to use sys.exit() in a way similar to C display > the error message first and invoke sys.exit() afterwards with a numerical > argument. > > -- oh ok , got it thanks :) -- https://mail.python.org/mailman/listinfo/python-list
python unit test frame work
Hello Team,
Iam on python 2.7 and linux. Iam trying to understand the python unit
test frame work and also trying to fix the below code , any help in
this regard would be appreciated ?
# Sample code starts here
inject_failure = {}
report = ""
ClassIsSetup = False
ClassCleanup = False
class Test_filesystem(unittest.TestCase):
""" """
def TestSetup(self):
""" Initial setup before unittests run """
logging.info("SETUP.Started !!!")
if not os.path.ismount("/tmp"): # Comment 1.
logging.error("Error: /tmp is not mounted")
sys.exit("/tmp is not mounted ...Exiting !!!")
if self.create_dataset() and capture_chksum():
try:
test01_log = os.path.join(LOG_DIR, "test01_corrupt.log")
self.inject_failure['test01'] = tool.run_tool(test01_log)
time.sleep(10)
test02_log = os.path.join(LOG_DIR, "test01_corrupt.log")
self.inject_failure['test01'] = tool.run_tool(test02_log)
time.sleep(10)
except Exception, e:
logging.error(e)
sys.exit(1)
if not run_scanner():
sys.exit(1)
else:
try:
self.__class__.report = report_tool()
except Exception, e:
logging.error(e)
sys.exit("Running Reporting tool failed")
logging.info("SETUP.Done !!!")
def setUp(self):
if not self.ClassIsSetup:
self.__class__.ClassIsSetup = True
self.setupClass()
def setupClass(self):
self.TestSetup()
def test_01_inode_test(self):
""" test01: """
logging.info("### Executing test01: inode corruption ###")
self.assertTrue(run_db_tool(self.__class__.report,
self.find_failure['test01']))
def test_02_hardlink_test(self):
""" test02: """
logging.info("### Executing test01: inode corruption ###")
self.assertTrue(run_db_tool(self.__class__.report,
def tearDown(self):
if self.ClassCleanup:
self.tearDownClass()
def tearDownClass(self):
self.cleanup()
# Comment 2
def cleanup(self):
""" Cleanup all the data & logs """
logging.info("Cleaning all data")
os.system("rm -rf /tmp/data_set")
def main():
unittest.main()
if __name__ == '__main__':
main()
# Sample code ends here
Questions :
1. If the setUp() fails the code still tries to run through the test
and assert's with error. How do I avoid these Error on the console ,
actually test01, test02, . etc , shouldn't run if the setup Failed ?
Example: If the mount fails i.e if not os.path.ismount("/tmp"): in
TestSetup(). we will get the below output:
#c_t.py
==
ERROR: test01: test_01_inode_test
--
Traceback (most recent call last):
File "c_t.py", line xx, in setUp
self.setupClass()
File "c_t.py", line xxx, in TestSetup
self.TestSetup()
File "c_t.py", line xx, in corruptSetup
sys.exit("/tmp is not mounted ...Exiting !!!")
SystemExit: /tmp is not mounted ...Exiting !!!
==
ERROR: test02
------
Traceback (most recent call last):
File "c_t.py", line 162, in test_02_hardlink_test
self.inject_failures['test02']))
KeyError: 'test02'
Ran 2 tests in 0.003s
FAILED (errors=2)
2. The cleanup() never gets executed at the end of the test.
3. Any better idea or suggestions to improve the above code ?
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: python unit test frame work
+python list . sorry I accidentally did a reply to Peter. On Dec 11, 2015 3:57 AM, "Ganesh Pal" wrote: > > > > Drop the habit to sprinkle sys.exit() all over the place. A well-behaved > > application has one exit point, at the end of the main module. > I was using sys.exit() as the means to stop the execution or terminate the program. I can't think of an alternative for my case : I have multiple checks if I don't meet them continuing with the main program doesn't make sense Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
Re: python unit test frame work
On Thu, Dec 10, 2015 at 9:20 PM, Peter Otten <__pete...@web.de> wrote:
> Ganesh Pal wrote:
>
> I recommend that you reread the unittest documentation.
>
> setUpClass() should be a class method, and if it succeeds you can release
> the ressources it required in the corresponding tearDownClass() method. As
> written the flags and the setUp()/tearDown() seem unnecessary.
>
Thanks to peter , Cameron and Ben Finney , for replying to my various
question post . I needed a hint on the below
1. If there is a setUpClass exception or failure , I don't want the
unittest to run ( I don't have teardown ) how do I handle this ?
The traceback on the console looks very bad it repeats for all
the test cases , that means if I have 100 testcases if setup fails .
I will get the failure for all the test cases
#c_t.py
==
ERROR: test01: test_01_inode_test
--
Traceback (most recent call last):
File "c_t.py", line xx, in setUp
self.setupClass()
File "c_t.py", line xxx, in TestSetup
self.TestSetup()
File "c_t.py", line xx, in corruptSetup
sys.exit("/tmp is not mounted ...Exiting !!!")
SystemExit: /tmp is not mounted ...Exiting !!!
==
ERROR: test02
--
Traceback (most recent call last):
File "c_t.py", line 162, in test_02_hardlink_test
self.inject_failures['test02']))
KeyError: 'test02'
Ran 2 tests in 0.003s
FAILED (errors=2)
--
https://mail.python.org/mailman/listinfo/python-list
Calling a list of functions
Hi Team, Iam on linux and python 2.7 . I have a bunch of functions which I have run sequentially . I have put them in a list and Iam calling the functions in the list as shown below , this works fine for me , please share your opinion/views on the same Sample code : def print1(): print "one" def print2(): print "two" def print3(): print "three" print_test = [print1(),print2(),print3()] //calling the function for test in range(len(print_test)): try: print_test[test] except AssertionError as exc: Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
does the order in which the modules are placed in a file matters ?
Iam on python 2.7 and linux .I need to know if we need to place the modules in a particular or it doesn't matter at all order while writing the program For Example import os import shlex import subprocess import time import sys import logging import plaftform.cluster from util import run def main(): """ ---MAIN--- """ if __name__ == '__main__': main() In the above example : 1. Iam guessing may be the python modules like os , shlex etc come first and later the user defined modules like import plaftform.cluster .etc come latter Sorry if my question sounds dump , I was running pep8 and don't see its bothered much about it Regards, Ganesh -- https://mail.python.org/mailman/listinfo/python-list
How to ignore error with anon-zero exit status
def run_scanner():
"""
Mount /filesystems and run scanner
"""
for cmd in [" mount /filesystems ", " scanner_start"]:
try:
out, err, ret = run(cmd, timeout=3600)
if ret != 0:
logging.error("Can't run %s got %s (%d)!" % (cmd, err, ret))
return False
except Exception as e:
logging.exception("Failed to run %s got %s" % (cmd, e))
return False
logging.info("Mount /tmp.Done !!!")
time.sleep(30)
Iam on python 2.6 and Linux , I need you inputs on how to ignore an
specific error when the mount fails
In general the mount has zero status if it succeeds and anon-zero exit
status if it fails.
1.But for one rare case the mount succeeds but returns anon-zero exit
status and when we get “Reading GUID from da0xxx: No such file or
directory” error , how to ignore this error and proceed with the above
code
2. Also need to add this check only for mount case i.e mount
/filesystems and not scanner_start
Any recommendations would be appreciated.
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Ignore error with non-zero exit status (was: How to ignore error with anon-zero exit status)
> (Polite people would *ask* a *question*.) I am a polite person , sorry if the wording was harsh. > (“_a non-zero_”, with a space in-between. “anon” can be misunderstood as an > abbreviation for “anonymous”.) It was a typo. > Most simple solution for this: Do not use a loop. More "complicated" > solution: Use an “if” statement. I want to stick on to loop and try modifying the code with if. Should I use some comparison operation with if ? -- https://mail.python.org/mailman/listinfo/python-list
Re: does the order in which the modules are placed in a file matters ?
Thanks to Don , Chris and Carl for sharing your view on this topic . -- https://mail.python.org/mailman/listinfo/python-list
python unit test framework sample code
Hello Team,
I have written a small program using python unit test framework . I
need your guidance to find out
1. If I have used the fixtures and classes properly ( first oop program) :) )
2. why does unittest2.SkipTest not print the message when the failures
are encountered ?
3. Also sys.stderr.write("Test run failed ({e})".format(e=e) ) does
not display error on failure ?
4. Any other general comment's
Iam on Python 2.6 and using Linux
Sample code:
class FileSystemTest(unittest2.TestCase):
block_address = {}
report = ""
@classmethod
def setUpClass(cls):
cls.FileSystemSetup()
@classmethod
def FileSystemSetup(cls):
"""
Initial setup before unittest is run
"""
logging.info("SETUP.Started !!!")
try:
inode_01 =
corrupt.get_lin(os.path.join(CORRUPT_DIR,"inode_lin.txt"))
test_01_log = os.path.join(LOG_DIR, "test_01_corrupt.log")
cls.block_address['test_01'] =
corrupt.inject_corruption(test_01_log,
lin=inode_01, object="inode", offset="18",
size="4", optype="set")
time.sleep(10)
except Exception, e:
logging.error(e)
raise unittest2.SkipTest("class setup failed")
if not corrupt.run_scanner():
raise unittest2.SkipTest("class setup failed")
else:
try:
cls.report = corrupt.run_report()
except Exception, e:
logging.error(e)
raise unittest2.SkipTest("class setup failed")
logging.info("SETUP.Done !!!")
def inode_corruption(self):
""" test_01: inode corruption """
self.assertTrue(corrupt.run_query_tool(self.__class__.report,
self.block_address['test_01']))
@classmethod
def tearDownClass(cls):
cls.tearDown()
@classmethod
def tearDown(cls):
print "Entered tearDown()"
try:
cls.cleanup = cleanup()
except Exception, e:
logging.error(e)
def main():
""" ---MAIN--- """
global LOG_DIR
try:
if len(sys.argv) > 1:
LOG_DIR = str(sys.argv[1])
except Exception, e:
print(USAGE)
return errno.EINVAL
functions = [create_logdir,create_dataset,corrupt.prep_cluster]
for func in functions:
try:
func()
except Exception, e:
logging.error(e)
sys.stderr.write("Test run failed ({e})".format(e=e))
unittest2.main()
if __name__ == '__main__':
main()
PS : Happy New year Wishes to all the form members !!
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Fwd: python unit test framework sample code
Apologies, looks like I did a reply instead of reply-all. So
forwarding this email , please provide guidance on the same
-- Forwarded message --
From: Ganesh Pal
Date: Thu, Jan 7, 2016 at 12:26 PM
Subject: Re: python unit test framework sample code
To: Terry Reedy
> Unless you have a large program already in 2.6 and are just adding tests
> (perhaps so you can more easily upgrade someday), consider upgrading to a
> newer version.
Sure , but for now Iam limited to use 2.6 , hence I cant upgrade need
to work with 2.6
>>
>> class FileSystemTest(unittest2.TestCase):
>> block_address = {}
>> report = ""
>>
>> @classmethod
>> def setUpClass(cls):
>> cls.FileSystemSetup()
>
>
> This is senseless. Put the body of FileSystemSetup here.
I didn't understand which line is senseless. I was trying to refer
this example on stack overflow
http://stackoverflow.com/questions/5938517/not-able-call-a-local-method-from-setupclass
class TestSystemPromotion(unittest2.TestCase):
@classmethod
def setUpClass(cls):
cls.setup_test_data()
@classmethod
def setup_test_data(cls):
...
def test_something(self):
...
class FileSystemTest(unittest2.TestCase):
block_address = {}
report = ""
@classmethod
def setUpClass(cls):
cls.FileSystemSetup()
@classmethod
def FileSystemSetup(cls):
"""
Initial setup before unittest is run
"""
logging.info("SETUP.Started !!!")
>> def inode_corruption(self):
>> """ test_01: inode corruption """
>> self.assertTrue(corrupt.run_query_tool(self.__class__.report,
>
>
> self.block_address['test_01']))
>
> Assuming that unittest2 is same as unittest, test methods must be called
> test_xyz. So this is not run, hence no error.
Sorry I have changed this from inode_corruption' to 'test_inode_corruption
>
>> @classmethod
>> def tearDownClass(cls):
>> cls.tearDown()
>
>
> Ditto. Put real body here.
>
>> @classmethod
>> def tearDown(cls):
>
>
> The above refers to functions you did not post. For current unittest, it
> looks likes you should be using a setUpModule (possible tearDownModule)
> functions, but I don't know if those are available in the older unittest2.
we have tearDownClass and setUpClass in python 2.6 and under unittest2
>>> help(unittest2.TestCase.tearDownClass)
Help on method tearDownClass in module unittest2.case:
tearDownClass(cls) method of __builtin__.type instance
Hook method for deconstructing the class fixture after running all
tests in the class.
>>> help(unittest2.TestCase.setUpClass)
Help on method setUpClass in module unittest2.case:
setUpClass(cls) method of __builtin__.type instance
Hook method for setting up class fixture before running tests in the class.
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: python unit test framework sample code
Totally stuck with this
On Jan 10, 2016 7:11 PM, "Ganesh Pal" wrote:
> Apologies, looks like I did a reply instead of reply-all. So
> forwarding this email , please provide guidance on the same
>
> -- Forwarded message ------
> From: Ganesh Pal
> Date: Thu, Jan 7, 2016 at 12:26 PM
> Subject: Re: python unit test framework sample code
> To: Terry Reedy
>
> > Unless you have a large program already in 2.6 and are just adding tests
> > (perhaps so you can more easily upgrade someday), consider upgrading to a
> > newer version.
>
> Sure , but for now Iam limited to use 2.6 , hence I cant upgrade need
> to work with 2.6
>
> >>
> >> class FileSystemTest(unittest2.TestCase):
> >> block_address = {}
> >> report = ""
> >>
> >> @classmethod
> >> def setUpClass(cls):
> >> cls.FileSystemSetup()
> >
> >
> > This is senseless. Put the body of FileSystemSetup here.
>
> I didn't understand which line is senseless. I was trying to refer
> this example on stack overflow
>
> http://stackoverflow.com/questions/5938517/not-able-call-a-local-method-from-setupclass
>
>
> class TestSystemPromotion(unittest2.TestCase):
>
> @classmethod
> def setUpClass(cls):
> cls.setup_test_data()
>
>
> @classmethod
> def setup_test_data(cls):
> ...
>
> def test_something(self):
> ...
>
>
> class FileSystemTest(unittest2.TestCase):
> block_address = {}
> report = ""
>
> @classmethod
> def setUpClass(cls):
> cls.FileSystemSetup()
>
> @classmethod
> def FileSystemSetup(cls):
> """
> Initial setup before unittest is run
> """
> logging.info("SETUP.Started !!!")
>
> >> def inode_corruption(self):
> >> """ test_01: inode corruption """
> >> self.assertTrue(corrupt.run_query_tool(self.__class__.report,
> >
> >
> > self.block_address['test_01']))
> >
> > Assuming that unittest2 is same as unittest, test methods must be called
> > test_xyz. So this is not run, hence no error.
>
> Sorry I have changed this from inode_corruption' to 'test_inode_corruption
>
> >
> >> @classmethod
> >> def tearDownClass(cls):
> >> cls.tearDown()
> >
> >
> > Ditto. Put real body here.
> >
> >> @classmethod
> >> def tearDown(cls):
> >
> >
>
> > The above refers to functions you did not post. For current unittest, it
> > looks likes you should be using a setUpModule (possible tearDownModule)
> > functions, but I don't know if those are available in the older
> unittest2.
>
>
> we have tearDownClass and setUpClass in python 2.6 and under unittest2
>
> >>> help(unittest2.TestCase.tearDownClass)
> Help on method tearDownClass in module unittest2.case:
>
> tearDownClass(cls) method of __builtin__.type instance
> Hook method for deconstructing the class fixture after running all
> tests in the class.
>
> >>> help(unittest2.TestCase.setUpClass)
> Help on method setUpClass in module unittest2.case:
>
> setUpClass(cls) method of __builtin__.type instance
> Hook method for setting up class fixture before running tests in the
> class.
>
> Regards,
> Ganesh
>
--
https://mail.python.org/mailman/listinfo/python-list
TypeError: not all arguments converted during string formatting
Hi Team,
Iam on python 2.6 and Linux , I had replaced print out, err ret with
logging.info(out, err ,ret) in the below code . I am getting
"TypeError: not all arguments converted during string formatting"
error any quick suggestion
try:
out, err, ret = run(cmd, timeout=60)
# New line added below
logging.info(out, err ,ret)
if ret != 0:
logging.error("Can't run %s got %s (%d)!" % (cmd, err, ret))
raise Exception("Preparing cluster failed...Exiting !!!")
except Exception as e:
logging.exception("Failed to run %s got %s" % (cmd, e))
sys.exit("Preparing cluster failed")
logging.info("Preparing Cluster.Done !!!")
(Pdb) c
Traceback (most recent call last):
File "/usr/local/lib/python2.6/logging/__init__.py", line 755, in emit
File "/usr/local/lib/python2.6/logging/__init__.py", line 637, in format
File "/usr/local/lib/python2.6/logging/__init__.py", line 425, in format
File "/usr/local/lib/python2.6/logging/__init__.py", line 295, in getMessage
TypeError: not all arguments converted during string formatting
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: TypeError: not all arguments converted during string formatting
I think logging.info(out) works the problem is when I add
logging.info(out,err,ret) ,may be there is a better way to supply this
arguments
On Wed, Feb 17, 2016 at 7:28 PM, Ganesh Pal wrote:
> Hi Team,
>
>
> Iam on python 2.6 and Linux , I had replaced print out, err ret with
> logging.info(out, err ,ret) in the below code . I am getting
>
> "TypeError: not all arguments converted during string formatting"
> error any quick suggestion
>
>
> try:
> out, err, ret = run(cmd, timeout=60)
> # New line added below
> logging.info(out, err ,ret)
> if ret != 0:
> logging.error("Can't run %s got %s (%d)!" % (cmd, err, ret))
> raise Exception("Preparing cluster failed...Exiting !!!")
> except Exception as e:
> logging.exception("Failed to run %s got %s" % (cmd, e))
> sys.exit("Preparing cluster failed")
> logging.info("Preparing Cluster.Done !!!")
>
>
> (Pdb) c
> Traceback (most recent call last):
> File "/usr/local/lib/python2.6/logging/__init__.py", line 755, in emit
> File "/usr/local/lib/python2.6/logging/__init__.py", line 637, in format
> File "/usr/local/lib/python2.6/logging/__init__.py", line 425, in format
> File "/usr/local/lib/python2.6/logging/__init__.py", line 295, in getMessage
> TypeError: not all arguments converted during string formatting
>
>
> Regards,
> Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: TypeError: not all arguments converted during string formatting
On Wed, Feb 17, 2016 at 7:32 PM, Chris Angelico wrote:
> The print statement/function happily accepts multiple arguments, and
> will join them according to a set of predefined rules. The logging
> functions don't have those rules, so they take one message and some
> optional parameters. Try this, instead:
>
> logging.info("%r %r %r", out, err, ret)
>
Thanks this solved my issue :)
--
https://mail.python.org/mailman/listinfo/python-list
Python unittest2.SkipTest and general suggestion
Hello team,
Please provide your guidance on how to proceed with the below test ,
Iam on python 2.6 and Linux. I have a linitation to use python 2.6 and
unittest2
try:
import unittest2 as unittest
except ImportError:
import unittest
class isiCorruptTest(unittest.TestCase):
corrupt_baddr = {}
report= ""
@classmethod
def setUpClass(cls):
cls.corruptSetup()
@classmethod
def corruptSetup(cls):
"""
Initial setup before unittest run
"""
logging.info("SETUP.Started !!!")
try:
logging.info("Capturing data as part of prechecks for test_xx")
capture_data()
except Exception as e:
logging.error(e)
sys.exit("Corruption injection failed...Exiting !!!")
try:
corrupt.run_scan()
except Exception as e:
logging.error(e)
raise unittest.SkipTest("Failure running Integrity Scan ")
try:
cls.report = corrupt.run_report()
except Exception as e:
logging.error(e)
raise unittest.SkipTest("Failure running Reporting Tool ")
#sys.exit(1)
logging.info("SETUP.Done !!!")
def test_04_inode(self):
""" test04: """
logging.info("### Executing test04: ###")
self.assertTrue(corrupt.run_query(self.__class__.report,
self.corrupt_baddr['test04']))
def main():
""" ---MAIN--- """
# both function are not shown in this code
functions = [create_logdir, create_dataset ]
for func in functions:
try:
func()
except Exception as e:
logging.error(e)
return False
unittest.main()
if __name__ == '__main__':
main()
I have the below problems while running this code:
1. unittest.SkipTest does not the display the exception message that
is caught.
Example : The function corrupt.run_scan() returns none and False for failures
3. how do I the ensure that test_04 is run only it setUpClass succeeds ?
Example: Iam getting this error message for each tests on the console.
==
ERROR: test_04_inode (__main__.isiCorruptTest)
test04: inode offset, size corruption
--
Traceback (most recent call last):
File "c_t1.py", line 251, in test_04_inode_offset_size_corruption
self.corrupt_baddr['test04']))
KeyError: 'test04'
3. Any other suggestion welcome
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
can try expect have if else.
Hi Team,
Iam on python 2.6 , need input on the below piece of code.
EXIT_STATUS_ERROR=1
def create_dataset():
"""
"""
logging.info("Dataset create.Started !!!")
try:
if os.path.ismount("/nfs_mount"):
touch_file("inode_fixcrc.txt")
logging.info("Dataset create.Done !!!")
else:
raise Exception("/nfs_mount is not mounted. Dataset create
failed !!!")
return False
except Exception as e:
logging.error(e)
sys.stderr.write("Dataset create failed...Exiting !!!")
sys.exit(EXIT_STATUS_ERROR)
return True
1. Can we have if else with in a try except block
2. How can the above code be improved
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: can try expect have if else.
On Sun, Feb 21, 2016 at 10:37 PM, Ben Finney wrote:
> What result do you get when running that code? What empirical reason do
> you have to think it would work or not work?
I wanted to know was is it good to have if else with in a try expect
block , I was checking more from the programming perspective and if
I need to avoid it and if its recommended
>> 2. How can the above code be improved
>
> The following sequence of statements::
>
> raise Exception("/nfs_mount is not mounted. Dataset create failed !!!")
> return False
>
> do not make sense. The ‘return‛ statement will never be reached. So your
> intent must be something other than what is expressed by that code.
>
I was using it with create_data function in the below code . If
create_dataset failed the exception would be caught , I think you are
right I need not worry about returning False
functions = [create_logdir, create_dataset]
for func in functions:
try:
func()
except Exception as e:
logging.error(e)
return False
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python unittest2.SkipTest and general suggestion
On Sun, Feb 21, 2016 at 10:33 PM, Ben Finney wrote:
> You are already supplying a custom message to ‘self.skipTest’::
>
> except Exception as exc:
> logging.error(exc)
> raise unittest.SkipTest("Failure running Integrity Scan ")
>
> So you can change that message by including the text representation of
> the exception object::
>
> except Exception as exc:
> logging.error(exc)
> raise unittest.SkipTest(
> "Failure running Integrity Scan: {exc}".format(exc=exc))
>
Thank you for the pointers , but I modified my code to use raise
instead of exception , just wondering how will I display the message
with the below code ,
Sample code :
import os
try:
import unittest2 as unittest
except ImportError:
import unittest
class MyTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
#x = False
x = None
if not x:
raise unittest.SkipTest(
"class setup failed")
def test_one(self):
print "test_one"
def test_two(self):
print "test_two"
if __name__ == "__main__":
unittest.main()
output:
gpal-3c6dc81-1# python c_4.py
s
--
Ran 0 tests in 0.000s
OK (skipped=1)
--
https://mail.python.org/mailman/listinfo/python-list
How to remove the line numbers from the file in python
what would be the easiest way to remove the lines in the leading
numbers 1.e 1 ,2, 19 from this file using python ?
1 import os
2 Suite = "Test Mail"
3
4 def sendMail(x):
5 text = x
6 sendmail_location = "/home/prasad/onefs/share/sendmail" #
sendmail location
7 p = os.popen("%s -t" % sendmail_location, "w")
8 p.write("From: %s\n" % "ganesh@gmail.com")
9 p.write("To: %s\n" % "ganesh@gmail.com")
10 #p.write("To: %s\n" % "umamaheshwa...@gmail.com")
11 p.write("Subject: Suite : %s \n" % (Suite))
12 p.write("\n") # blank line separating headers from body
13 p.write("%s" %text)
14 status = p.close()
15
16 if status != 0:
17 print "Sendmail exit status", status
18
19 sendMail("Test Mail")
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to remove the line numbers from the file in python
Thanks it works fine :)
On Fri, Feb 26, 2016 at 5:01 PM, Peter Heitzer
wrote:
> Ganesh Pal wrote:
>>what would be the easiest way to remove the lines in the leading
>>numbers 1.e 1 ,2, 19 from this file using python ?
> import sys,re
> for line in sys.stdin:
> print re.sub('^\d+','',line).rstrip()
>
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
list index out of range Error , need to fix it or ignore it
Iam on python 2.6 and Linux , I need input on the below program ,
here is the spinet of my program
filename='/tmp2/2.txt'
def check_file():
"""
Run the command parallel on all the machines , if there is a
file named /tmp/file2.txt extract file2.txt
"""
global filename
baddr = ''
cmd = ("run_al_paral 'ls -al %s'" % (filename))
print(cmd)
stdout, stderr, exitcode = run(cmd)
print(stdout)
lines = stdout.strip().split('\n')
print(lines)
for line in lines:
if 'exited' in lines:
continue
file = lines[0].split()[9][6:]
break
print file
return file
def main():
functions = [check_file]
for func in functions:
try:
func()
except Exception as e:
return False
if __name__ == '__main__':
main()
1.If the file is present in any one of the machine the program works
fine , example if the file is in machine4 it works fine,
machine-4# touch /tmp2/2.txt
machine-4# python c_4.py
run_al_parall 'ls -al /tmp2/2.txt'
machine-4: -rw-r--r-- 1 root wheel 0 Feb 27 08:15 /tmp2/2.txt
gpal-machine-2 exited with status 1
gpal-machine-5 exited with status 1
gpal-machine-3 exited with status 1
gpal-machine-1 exited with status 1
['machine-4: -rw-r--r-- 1 root wheel 0 Feb 27 08:15 /tmp2/2.txt',
'gpal-machine-2 exited with status 1', 'gpal-machine-5 exited with
status 1', 'gpal-machine-3 exited with status 1', 'gpal-machine-1
exited with status 1']
2.txt
2. But if the file is not present we get index out of range error , do
we need to fix this or its expected ? or its ok.
machine-4# python c_4.py
isi_for_array 'ls -al /tmp2/2.txt'
machine-2 exited with status 1
machine-1 exited with status 1
machine-4 exited with status 1
machine-5 exited with status 1
machine-3 exited with status 1
['machine-2 exited with status 1', 'machine-1 exited with status 1',
'machine-4 exited with status 1', 'machine-5 exited with status 1',
'machine-3 exited with status 1']
ERROR:root:list index out of range
3. Any other tips to improve the program
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: list index out of range Error , need to fix it or ignore it
changed baddr="" to file ="" in the example program , sorry for the typo
> filename='/tmp2/2.txt'
>
> def check_file():
> """
> Run the command parallel on all the machines , if there is a
> file named /tmp/file2.txt extract file2.txt
>
> """
> global filename
> file = ''
> cmd = ("run_al_paral 'ls -al %s'" % (filename))
> print(cmd)
> stdout, stderr, exitcode = run(cmd)
> print(stdout)
> lines = stdout.strip().split('\n')
> print(lines)
> for line in lines:
> if 'exited' in lines:
> continue
>
> file = lines[0].split()[9][6:]
> break
> print file
> return file
>
> def main():
> functions = [check_file]
> for func in functions:
> try:
> func()
> except Exception as e:
> return False
> if __name__ == '__main__':
> main()
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: list index out of range Error , need to fix it or ignore it
>>
> what is run(...)
>
The run (_ is a wrapper it uses suprocess.Popen and returns stdout
,error and extitcod e
> not a good idea to have catchall exception
how to fix this ?
>
>> > return False
>> > if __name__ == '__main__':
>> > main()
>> >
>> --
>>
> copy and paste your traceback
>
I get " ERROR:root:list index out of range" error if the file is
not found in any of the machine , I dont have a trackback back
def check_file():
"""
Run the command parallel on all the machines , if there is a
file named /tmp/file2.txt extract file2.txt
"""
global filename
file = ''
cmd = ("run_al_paral 'ls -al %s'" % (filename))
print(cmd)
stdout, stderr, exitcode = run(cmd)
print(stdout)
lines = stdout.strip().split('\n')
print(lines)
for line in lines:
if 'exited' in lines:
continue
file = lines[0].split()[9][6:]
break
print file
return file
def main():
functions = [check_file]
for func in functions:
try:
func()
except Exception as e:
return False
if __name__ == '__main__':
main()
1. But if the file is not present we get index out of range error , do
we need to fix this or its expected ? or its ok.
machine-4# python c_4.py
isi_for_array 'ls -al /tmp2/2.txt'
machine-2 exited with status 1
machine-1 exited with status 1
machine-4 exited with status 1
machine-5 exited with status 1
machine-3 exited with status 1
['machine-2 exited with status 1', 'machine-1 exited with status 1',
'machine-4 exited with status 1', 'machine-5 exited with status 1',
'machine-3 exited with status 1']
ERROR:root:list index out of range
3. Any other tips to improve the program
--
https://mail.python.org/mailman/listinfo/python-list
usage of try except for review.
Iam on python 2.6 and Linux , need your suggestion on the usage of try
and except in this program and
Modified code:
#!/usr/bin/env python
"""
"""
import os
import shlex
import subprocess
import sys
import time
import logging
import run
import pdb
def run_cmd_and_verify(cmd, timeout=1000):
try:
out, err, ret = run(cmd, timeout=timeout)
assert ret ==0,"ERROR (ret %d): " \
" \nout: %s\nerr: %s\n" % (ret, out, err)
except Exception as e:
logging.error("Failed to run %s got %s" % (cmd, e))
return False
return True
def run_test():
"""
Mount
"""
pdb.set_trace()
for cmd in ["mount /nfs_mount1", "mount /cifs_mount1"]:
try:
if not run_cmd_and_verify(cmd, timeout=3600):
return False
except:
logging.error("Failure while running command %")
logging.info("Setup and Creation Done !!!")
#
cmd = "run_scan"
out, err, ret = run(cmd)
for cmd in ["create_data.py -nfs ",
"validate.py -30 "]:
try:
if not run_cmd_and_verify(cmd, timeout=3600):
return False
except:
logging.error("")
return False
logging.info("Mount IS START.Done !!!")
def main():
if not run_test():
sys.exit("Exiting Main")
if __name__ == '__main__':
main()
Question 1:
1. Have I used try and expect block correctly ? , In my case I have
the except block that's is not needed it just gives an message I
have still included for the sake of try block
try:
if not run_cmd_and_verify(cmd, timeout=3600):
return False
except:
logging.error("inside except")
return False
2. If a failure’s are encountered the error by assert condition the
errors are now displayed on the screen , how do I redirect it to log
file using logging error
def run_cmd_and_verify(cmd, timeout=1000):
try:
out, err, ret = run(cmd, timeout=timeout)
assert ret ==0,"ERROR (ret %d): " \
" \nout: %s\nerr: %s\n" % (ret, out, err)
except Exception as e:
logging.error("Failed to run %s got %s" % (cmd, e))
return False
return True
#script_10.py
Failed to run mount /nfs got ERROR (ret 1):
out:
host-44-3 exited with status 1
err:
host-44-3: mount_efs: on /nfs: efs is already mounted
3. my function def has 1000 but Iam using 3600 in the calling fnx etc
, Time out value are overwritten ?
4. Any further improvement particularly on try -except ?
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: usage of try except for review.
Iam really sorry , I will have to resend my question again , because
the earlier post had mistakes and formatting was bad , so apologies
for top posting will try to avoid such mistakes in future.
Iam on python 2.6 and Linux , need your suggestion on the usage of try
and except in this program
#!/usr/bin/env python
"""
"""
import os
import shlex
import subprocess
import sys
import time
import logging
import run
import pdb
def run_cmd_and_verify(cmd, timeout=1000):
try:
out, err, ret = run(cmd, timeout=timeout)
assert ret ==0,"ERROR (ret %d): " \
" \nout: %s\nerr: %s\n" % (ret, out, err)
except Exception as e:
logging.error("Failed to run %s got %s" % (cmd, e))
return False
return True
def run_test():
"""
Mount
"""
pdb.set_trace()
for cmd in ["mount /nfs_mount1", "mount /cifs_mount1"]:
try:
if not run_cmd_and_verify(cmd, timeout=3600):
return False
except:
logging.error("Some meaningful message")
logging.info("Setup and Creation Done !!!")
#
cmd = "run_scan"
out, err, ret = run(cmd)
for cmd in ["create_data.py -nfs ",
"validate.py -30 "]:
try:
if not run_cmd_and_verify(cmd, timeout=3600):
return False
except:
logging.error("some meaningful message")
return False
logging.info("Mount IS START.Done !!!")
def main():
if not run_test():
sys.exit("Exiting Main")
if __name__ == '__main__':
main()
Question 1:
(a) Have I used try and expect block correctly ? in run_test() I
have expect block which displays some error message instead of pass
, is it fine ?
(b) Have I returned True and False correctly , looks fine for me
try:
if not run_cmd_and_verify(cmd, timeout=3600):
return False
except:
logging.error("inside except")
return False
Question 2.
(a) If a failure’s are encountered the error by assert condition the
errors are now displayed on the screen , how do I redirect it to log
file using logging error
because the moment assert ret==0 becomes true the program exits
def run_cmd_and_verify(cmd, timeout=1000):
try:
out, err, ret = run(cmd, timeout=timeout)
assert ret ==0,"ERROR (ret %d): " \
" \nout: %s\nerr: %s\n" % (ret, out, err)
except Exception as e:
logging.error("Failed to run %s got %s" % (cmd, e))
return False
return True
#script_10.py
Failed to run mount /nfs got ERROR (ret 1):
out:
host-44-3 exited with status 1
err:
host-44-3: mount_efs: on /nfs: efs is already mounted
3. my function def has 1000 but Iam using 3600 in the calling fnx etc
, Time out value are overwritten ?
4. Any further improvement particularly on try -except ?
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
common mistakes in this simple program
Iam on python 2.6 , need inputs on the common mistakes in this program
, may be you suggest what need to be improved from
1. usage of try- expect
2. Return of True/ False
3. Other improvement
#!/usr/bin/env python
"""
"""
import os
import shlex
import subprocess
import sys
import time
import logging
import run
import pdb
def run_cmd_and_verify(cmd, timeout=1000):
try:
pdb.set_trace()
out, err, ret = run(cmd, timeout=timeout)
assert ret ==0,"ERROR (ret %d): " \
" \nout: %s\nerr: %s\n" % (ret, out, err)
except Exception as e:
print("Failed to run %s got %s" % (cmd, e))
return False
return True
def prep_host():
"""
Prepare clustering
"""
for cmd in ["ls -al",
"touch /tmp/file1",
"mkdir /tmp/dir1"]:
try:
if not run_cmd_and_verify(cmd, timeout=3600):
return False
except:
print("Error: While preparing cluster !!!")
return False
print("Preparing Cluster.Done !!!")
return True
def main():
functions = [prep_host]
for func in functions:
try:
func()
except Exception as e:
print(e)
return False
if __name__ == '__main__':
main()
Regards,
Gpal
--
https://mail.python.org/mailman/listinfo/python-list
Re: common mistakes in this simple program
On Mon, Feb 29, 2016 at 9:59 PM, Ian Kelly wrote:
> On Mon, Feb 29, 2016 at 8:18 AM, Ganesh Pal wrote:
>> Iam on python 2.6
>> 1. usage of try- expect
>
> try-except in every single function is a code smell. You should only
> be using it where you're actually going to handle the exception. If
> you catch an exception just to log it, you generally should also
> reraise it so that something further up the call chain has the
> opportunity to handle it.
How do we reraise the exception in python , I have used raise not
sure how to reraise the exception
>
>> def run_cmd_and_verify(cmd, timeout=1000):
>> try:
>> pdb.set_trace()
>> out, err, ret = run(cmd, timeout=timeout)
>
> What is "run"? It's imported like a module above, but here you're
> using it like a function.
Sorry run is a function which was imported from a library , the
function had to be # from utility import run ,
>> assert ret ==0,"ERROR (ret %d): " \
>> " \nout: %s\nerr: %s\n" % (ret, out, err)
>> except Exception as e:
>> print("Failed to run %s got %s" % (cmd, e))
>> return False
>> return True
>>
>> def prep_host():
>> """
>> Prepare clustering
>> """
>> for cmd in ["ls -al",
>> "touch /tmp/file1",
>> "mkdir /tmp/dir1"]:
>> try:
>> if not run_cmd_and_verify(cmd, timeout=3600):
>> return False
>> except:
>
> What exceptions are you expecting this to catch? run_cmd_and_verify
> already catches any expected exceptions that it raises.
This is a wrapper for Popen it runs the command and returns stdout
,stderror and returncode .The only exception it can return is a time
out exception
Here is the modified buggy code , Can I have Try and except with Pass
, how do I modify the try and expect in the pre-host ?
#!/usr/bin/env python
"""
bugging code
"""
import logging
from utility import run
def run_cmd_and_verify(cmd, timeout=1000):
try:
out, err, ret = run(cmd, timeout=timeout)
assert ret ==0,"ERROR (ret %d): " \
" \nout: %s\nerr: %s\n" % (ret, out, err)
except Exception as e:
logging.error("Failed to run %s got %s" % (cmd, e))
return False
return True
def prep_host():
"""
Prepare clustering
"""
for cmd in ["ls -al",
"touch /tmp/file1",
"mkdir /tmp/dir1"]:
try:
if not run_cmd_and_verify(cmd, timeout=3600):
logging.info("Preparing cluster failed ...")
return False
except:
pass
logging.info("Preparing Cluster.Done !!!")
return True
def main():
functions = [prep_host]
for func in functions:
try:
func()
except Exception as e:
logging.info(e)
return False
if __name__ == '__main__':
main()
Regards,
Gpal
--
https://mail.python.org/mailman/listinfo/python-list
