building strings from variables
Following a discussion with an associate at work about various ways to
build strings from variables in python, I'd like to hear your opinions
and preferred methods. The methods we discussed are:
1. some_string = "cd %s ; %s %d %s %s" % ( working_dir, ssh_cmd,
some_count, some_param1, some_param2)
2. import string
template = string.Template("cd $dir ; $cmd $count $param1
$param2")
some_string = template.substitute(dict(dir=working_dir,
cmd=ssh_cmd,
count=some_count,
pararm1=some_param1,
param2=some_param2))
here you can use a couple of nice tricks by using class.__dict__ and
globals() \ locals() dictionaries.
3. some_string = "cd "+working_dir+" ; "+ssh_cmd+ "
"+str(some_count)+" "+some_param1+" "+some_param2
(all these are supposed to produce the same strings)
Which methods do you know of \ prefer \ think is better because...?
I will appreciate any opinions about the matter.
--
http://mail.python.org/mailman/listinfo/python-list
Re: building strings from variables
Matthew Warren wrote:
> > -Original Message-
> > From:
> > [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED]
> > rg] On Behalf Of Gal Diskin
> > Sent: 05 October 2006 16:01
> > To: python-list@python.org
> > Subject: building strings from variables
> >
> > Following a discussion with an associate at work about various ways to
> > build strings from variables in python, I'd like to hear your opinions
> > and preferred methods. The methods we discussed are:
> > 1. some_string = "cd %s ; %s %d %s %s" % ( working_dir, ssh_cmd,
> > some_count, some_param1, some_param2)
> >
> > 2. import string
> > template = string.Template("cd $dir ; $cmd $count $param1
> > $param2")
> > some_string = template.substitute(dict(dir=working_dir,
> >
> > cmd=ssh_cmd,
> >
> > count=some_count,
> >
> > pararm1=some_param1,
> >
> > param2=some_param2))
> > here you can use a couple of nice tricks by using class.__dict__ and
> > globals() \ locals() dictionaries.
> >
> > 3. some_string = "cd "+working_dir+" ; "+ssh_cmd+ "
> > "+str(some_count)+" "+some_param1+" "+some_param2
> > (all these are supposed to produce the same strings)
> >
> > Which methods do you know of \ prefer \ think is better because...?
> > I will appreciate any opinions about the matter.
>
> :D
>
> I think, it would depend really on what your aims are (readable code,
> fast string generation...), and how the vars you want to build the
> string from are respresented in your code (is it natural to use a dict
> etc..)
>
> I kicked off a conversation similar to this earlier today, and that was
> my conclusion after helpful debate & advice.
>
> Matt.
>
>
> This email is confidential and may be privileged. If you are not the intended
> recipient please notify the sender immediately and delete the email from your
> computer.
>
> You should not copy the email, use it for any purpose or disclose its
> contents to any other person.
> Please note that any views or opinions presented in this email may be
> personal to the author and do not necessarily represent the views or opinions
> of Digica.
> It is the responsibility of the recipient to check this email for the
> presence of viruses. Digica accepts no liability for any damage caused by any
> virus transmitted by this email.
>
> UK: Phoenix House, Colliers Way, Nottingham, NG8 6AT UK
> Reception Tel: + 44 (0) 115 977 1177
> Support Centre: 0845 607 7070
> Fax: + 44 (0) 115 977 7000
> http://www.digica.com
>
> SOUTH AFRICA: Building 3, Parc du Cap, Mispel Road, Bellville, 7535, South
> Africa
> Tel: + 27 (0) 21 957 4900
> Fax: + 27 (0) 21 948 3135
> http://www.digica.com
Matt,
Thanks for replying. I tried looking up your discussion and I'm unsure
if I found the right post. Would you mind linking to it?
--
Gal Diskin
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Work: +972-4-865-1637
Cell: +972-54-7594166
--
http://mail.python.org/mailman/listinfo/python-list
Re: building strings from variables
First of all - thanks for all the replies. (Sorry for my slowness in
answering, but I wrote this message from work, just before leaving for
the weekend.)
I found a couple of posts that might be of interest:
Regarding speed I found a similar discussion in this newsgroup with
interesting results:
http://groups.google.com/group/comp.lang.python/browse_frm/thread/86aa0dd9dc0ed9cd/502a252f2c5ed778
Wesley, you might be interested to find that join is not necessarily
faster according to this thread. Jordan you might also be interested to
see their benchmarking.
The following post may also be of interest:
http://groups.google.com/group/comp.lang.python/msg/5ca4321e1d493fe9
(I haven't found the time to check this one myself so it comes with
limited warranty)
Regarding readability and "better" code my preferred methods are the 2
methods I haven't mentioned introduced by Rainy (%(varname)s ..." %
globals()) and by Scott David Daniels (' '.join[...]). Thanks, I
already knew join, but failed to include it for some reason but I
wasn't aware you could use dictionaries like this as well.
Thanks Jordan for making the effort to do the benchmarking. I think I
got similar results on several machines - according to my personal
benchmarking (based on short strings, and therefore relevant only to
such), the fastest method is "%s %d %s" % (p1, p2, p3) . However, all
other methods get close results (i.e. not above 1.5x runtime) except
for template.replace which is much slower by rate of about 15x. When it
comes to choosing between using template.replace or "%(varname)s ..." %
globals() my preference is for the latter without a doubt.
--
Gal Diskin
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Work: +972-4-865-1637
Cell: +972-54-7594166
--
http://mail.python.org/mailman/listinfo/python-list
Re: ultra newbie question (don't laugh)
John Salerno wrote: > Ok, I've decided to make a little project for myself which involves > storing employee information in an XML file. I'm doing this partly to > experiment with working with XML. The blocks in the file will look > something like this: > > >John >Salerno >United States >Texas >Houston >#etc... > > > I also plan to make a GUI frontend with wxPython for entering the > records. This will be fairly easy, but not as fun as writing the logic. > For now I've decided to focus just on writing the logic, and not worry > about the GUI yet. > > So this is what I came up with so far, then I sat staring at the screen > wondering how to proceed: > > class LabXMLWriter(object): > > def write_name(self, first, last, given=''): > > Suddenly I realized I just have no idea how to start thinking about what > I need to do. My first instinct was to use a class, as above, but then I > wondered if that was even necessary, since all I need to do is get > information from a user and write it to a file. Do I really need that > information stored in an object? > > Then I wondered if I needed an __init__ method, and what could go in it? > Or should I just make separate methods for each bit of information to > write to the file (i.e., name, birth location, address, phone number, > etc.). I thought maybe I could create the ID in the __init__ method > (salerjo01), but to do that I need the name first. I could do this: > > def __init__(self, first_name, last_name, given_name=''): > # code to initialize name and create ID > > But then I wondered if this detracts from the work that the class > methods would do later. > > So you see, what I'm asking for is very basic help, sort of along the > lines of "what things do I need to consider before I even begin this?" > Is OOP necessary here? Would utility functions work just as well for > simply writing the information to a file? > > Perhaps I should just take some kind of programming intro class! I read > all these Python books, but when it comes time to write something > non-trivial, I get stuck almost immediately with all the possibilities. > > Thanks, > John I think this is marely a matter of programming style in your case. If you like to use OOP use it. If not, don't. It doesn't seem to me that OOP is necessary to anything you just mentioned. Don't let the possibilities slow you - they're there so you can use whatever you like, not to flood you with too many options. Good luck, Gal -- http://mail.python.org/mailman/listinfo/python-list
threading, subprocesses and wait
Hi all, I'm writing a python program using threads to open several subprocesses concurrently (using module subprocess) and wait on them. I was wondering if there is a possibilty that a thread will return from wait even though the subprocess that finished was created by another thread thats also waiting. i.e. - 2 threads, each thread opens a subprocess and waits. The subprocess created by thread 1 has ended and thread 2 is released from wait instead of thread 1. Is this scenario possible? (I have reason to believe this has happend to me, but it seems unreasonable behaviour) Thanks in advance, Gals -- http://mail.python.org/mailman/listinfo/python-list
Re: threading, subprocesses and wait
Jean-Paul Calderone wrote: > On 26 Sep 2006 06:29:17 -0700, Gal Diskin <[EMAIL PROTECTED]> wrote: > >Hi all, > >I'm writing a python program using threads to open several subprocesses > >concurrently (using module subprocess) and wait on them. I was > >wondering if there is a possibilty that a thread will return from wait > >even though the subprocess that finished was created by another thread > >thats also waiting. > > > >i.e. - 2 threads, each thread opens a subprocess and waits. The > >subprocess created by thread 1 has ended and thread 2 is released from > >wait instead of thread 1. Is this scenario possible? (I have reason to > >believe this has happend to me, but it seems unreasonable behaviour) > > From a superficial reading of the code, it looks like this should not > happen. The wait call uses waitpid(2) which will only return when the > specified pid has terminated. > > On the other hand, it performs no error handling at all, so if the SIGCHLD > for thread 1's process is delivered to thread 2, it may cause the wait call > to fail with an EINTR OSError. Whether this is actually possible or not > depends on what signal handlers, if any, have been installed, as well as the > underlying C and threading libraries, though. > > Jean-Paul Thanks for the quick answer. -- http://mail.python.org/mailman/listinfo/python-list
Iterating over several lists at once
Hi, I am writing a code that needs to iterate over 3 lists at the same time, i.e something like this: for x1 in l1: for x2 in l2: for x3 in l3: print "do something with", x1, x2, x3 What I need to do is go over all n-tuples where the first argument is from the first list, the second from the second list, and so on... I was wondering if one could write this more easily in some manner using only 1 for loop. What I mean is something like this: for (x1,x2,x3) in (l1,l2,l3): print "do something with", x1, x2, x3 Or maybe like this: for x1 in l1, x2 in l2, x3 in l3: print "do something with", x1, x2, x3 However, this code obviously doesn't work... I'd be very happy to receive ideas about how to do this in one loop and with minimal initialization (if at all required). Thanks in advance, Gal -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterating over several lists at once
Nothing seriously wrong, but it's not too elegent. Especially when the number of lists you want to iterate over gets bigger (especially because of the indentation in python). As you noticed (an phrased better than me), what I was wondering is if there is a way to iterate over the cartesian product, but without actually doing all n for loops but using a single "for" loop. Thanks for replying me. On Dec 13, 3:58 pm, Roberto Bonvallet <[EMAIL PROTECTED]> wrote: > Gal Diskin wrote: > > Hi, > > I am writing a code that needs to iterate over 3 lists at the same > > time, i.e something like this: > > > for x1 in l1: > >for x2 in l2: > >for x3 in l3: > >print "do something with", x1, x2, x3What's wrong with this? > > [...] > > > I'd be very happy to receive ideas about how to do this in one loop and > > with minimal initialization (if at all required).def cartesian_product(l1, > > l2, l3): > for i in l1: > for j in l2: > for k in l3: > yield (i, j, k) > > for (i, j, k) in cartesian_product(l1, l2, l3): > print "do something with", i, j, k > > -- > Roberto Bonvallet -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterating over several lists at once
Thanks, that's an improvment (your first way). But I still wish I could find an even shorter (or more elegent) way of doing it. (Well, I guess if I expect every wish I have to come true I should at least wish for something more valuable.) Thanks again, Gal On Dec 13, 3:58 pm, "Fredrik Lundh" <[EMAIL PROTECTED]> wrote: > "Gal Diskin" wrote: > > I am writing a code that needs to iterate over 3 lists at the same > > time, i.e something like this: > > > for x1 in l1: > >for x2 in l2: > >for x3 in l3: > >print "do something with", x1, x2, x3 > > > What I need to do is go over all n-tuples where the first argument is > > from the first list, the second from the second list, and so on... > > > I was wondering if one could write this more easily in some manner > > using only 1 for loop. > > What I mean is something like this: > > > for (x1,x2,x3) in (l1,l2,l3): > >print "do something with", x1, x2, x3how about > > for x1, x2, x3 in func(l1, l2, l3): > print x1, x2, x3 > > where func is defined as, say, > > def func(l1, l2, l3): > return ((x1, x2, x3) for x1 in l1 for x2 in l2 for x3 in l3) > > or if you prefer > > def helper(l1, l2, l3): > for x1 in l1: > for x2 in l2: > for x3 in l3: > yield x1, x2, x3 > > -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterating over several lists at once
On Dec 13, 3:47 pm, "Gal Diskin" <[EMAIL PROTECTED]> wrote: > Hi, > I am writing a code that needs to iterate over 3 lists at the same > time, i.e something like this: > > for x1 in l1: > for x2 in l2: > for x3 in l3: > print "do something with", x1, x2, x3 > > What I need to do is go over all n-tuples where the first argument is > from the first list, the second from the second list, and so on... > > I was wondering if one could write this more easily in some manner > using only 1 for loop. > What I mean is something like this: > > for (x1,x2,x3) in (l1,l2,l3): > print "do something with", x1, x2, x3 > > Or maybe like this: > > for x1 in l1, x2 in l2, x3 in l3: > print "do something with", x1, x2, x3 > > However, this code obviously doesn't work... > > I'd be very happy to receive ideas about how to do this in one loop and > with minimal initialization (if at all required). > > Thanks in advance,Gal Sorry for bumping this thread up. I just got back from a vacation and I had to say thanks to all the people that took the time to answer me. To Maksim - that's not what I meant, I want to iterate over the cartesian product of the lists (all ordered tuples, _not_ the tuple created from the first item in each list, then the tubple created from the second item in each list... as your example does). Thanks, Gal -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess popen trouble
Your code seems correct to me.
Just as a long shot - have you tried setting bufsize=0 ?
nik wrote:
I am having trouble using subprocess popen and stdin/stdout
I have the following simple test application written in C++, that just
echoes back the stdin. It works fine running from the command line:
#include
#include
int main (int argc, char * const argv[]) {
int currenttemp = 0;
int settemp = 0;
char* str;
while(currenttemp < 500){
gets(str);
sscanf(str, ">%d", &settemp);
currenttemp = settemp;
printf("<%d\n", currenttemp++);
}
return 0;
}
##
Then I have the following python code. A listener thread that waits
for the output of test and the main part that creates the subprocess
Popen object, passes it to the thread and then sends user input to the
subprocess test.
import subprocess
import os, sys
from threading import *
import time
class TempXListen(Thread):
""" thread to listen for incomming
messages and put them onto a
shared queue, no time lost processing incoming lines
"""
def __init__(self, channel):
""" connect internal response queue
to external queue
"""
Thread.__init__(self)
self.setDaemon(True)
self._channel = channel
self.start()
def run(self):
""" listening loop
reads line off of input source
puts the line into the response Q shared with parent
thread
sets event that parent is waiting for to indicate Q has at
least 1 element
"""
while 1:
try:
response = self._channel.stdout.readline() #
read input line
print response
except Exception, e:
print 'Listener Exception: ' + str(e)
print "Temp Monitor Test"
tempx = subprocess.Popen('/Users/engineeringadmin/Documents/test/build/
Debug/test', \
bufsize=1, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, \
stderr=subprocess.PIPE)
print tempx.stderr.readline()
listener = TempXListen(tempx)
while 1:
data = sys.stdin.readline()
if data != '':
data = '>' + data + '\n'
tempx.stdin.write(data)
print data
time.sleep(1)
###
When I run it this is the output:
Temp Monitor Test
warning: this program uses gets(), which is unsafe.
45
45
The subprocess is opened, because the warning is coming off of the
test applications stderr, but nothing else seems to go in or out. I've
been looking at a lot of examples and a lot of different postings, and
can't see any mistakes in my use of subprocess. I would really
appreciate it if somebody could see where I am going wrong.
Thank you,
Nik
-- Gal Diskin
--
http://mail.python.org/mailman/listinfo/python-list
Re: profiling a C++ python extension
rasmus wrote: I have used gprof to profile stand alone C++ programs. I am also aware of pure python profilers. However, is there a way to get profile information on my C++ functions when they are compiled in a shared library (python extension module) and called from python. From what I can tell, gmon.out will not be generated unless the entire executable (python interpreter) was compiled with -pg. Is my only solution to recompile the python interpreter with -pg so that my extension module (also compiled with -pg) produces a gmon.out? Any suggestions or tips would be helpful. Matt This may not be the best solution but it will work without any recompiles. Use the binary instrumentation tool PIN from Intel (you can get it here: http://rogue.colorado.edu/pin/) as far as I understand the license it's free for non-commercial use (if you're looking for commercial use ask your legal department... :) ). Once you're using PIN all you need to do is write a simple "pintool" (a profiler) that will profile only your shared library and leave the rest of the code alone. (there is a mailing list called pinheads in yahoo that is monitored by the developers and a lot of example code in the released kits). Hope this helps, Gal Diskin -- http://mail.python.org/mailman/listinfo/python-list
