Re: Testing for performance regressions
On Mon, Apr 4, 2011 at 10:25 PM, Steven D'Aprano
wrote:
> On Mon, 04 Apr 2011 20:59:52 -0700, geremy condra wrote:
>
>> On Mon, Apr 4, 2011 at 7:45 PM, Steven D'Aprano
>> wrote:
>
>>> * The disclaimers about timing code snippets that can be found in the
>>> timeit module apply. If possible, use timeit rather than roll-you-own
>>> timers.
>>
>> Huh. In looking into timing attacks actually one of the biggest lessons
>> I learned was *not* to use timeit- that the overhead and variance
>> involved in using it will wind up consuming small changes in behavior in
>> ways that are fairly opaque until you really take it apart.
>
> Do you have more details?
>
> I follow the advice in the timeit module, and only ever look at the
> minimum value, and never try to calculate a mean or variance. Given the
> number of outside influences ("What do you mean starting up a browser
> with 200 tabs at the same time will affect the timing?"), I wouldn't
> trust a mean or variance to be meaningful.
I think it's safe to treat timeit as an opaque, medium-precision
benchmark with those caveats. If you need actual timing data though-
answering the question 'how much faster?' rather than 'which is
faster?' just taking actual timings seems to provide much, much better
answers. Part of that is because timeit adds the cost of the for loop
to every run- here's the actual code:
def inner(_it, _timer):
%(setup)s
_t0 = _timer()
for _i in _it:
%(stmt)s
_t1 = _timer()
return _t1 - _t0
(taken from Lib/timeit.py line 81)
where %(setup)s and %(stmt)s are what you passed in. Obviously, if the
magnitude of the change you're looking for is smaller than the
variance in the for loop's overhead this makes things a lot harder
than they need to be, and the whole proposition gets pretty dodgy for
measuring in the sub-millisecond range, which is where many timing
attacks are going to lay. It also has some problems at the opposite
end of the spectrum- timing large, long-running, or memory-intensive
chunks of code can be deceptive because timeit runs with the GC
disabled. This bit me a while back working on Graphine, actually, and
it confused the hell out of me at the time.
I'm also not so sure about the 'take the minimum' advice. There's a
reasonable amount of empirical evidence suggesting that timings taken
at the 30-40% mark are less noisy than those taken at either end of
the spectrum, especially if there's a network performance component.
YMMV, of course.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
replace regex in file using a dictionary
Hi,
So i want to replace multiple lines in a text file and I have reasoned
the best way to do this is with a dictionary. I have simplified my
example and broadly I get what I want however I am now printing my
replacement string and part of the original expression. I am guessing
that I will need to use a more complicated expression than I am
currently am to achieve what I want?
my aim is to match the expression before the "=" and replace the
string after the equals...
My failed example...
def replace_keys(text, replacements_dict):
for key, var in replacements_dict.iteritems():
replacement = "%s = %s" % (key, var)
text = text.replace(key, replacement)
return text
# example of two lines of the file
str = \
"""structcn = 150.0
metfrac0 = 0.85
"""
# replacements
replacement_dict = {"structcn": "999.0", "metfrac0": "0.85"}
new_str = replace_keys(str, replacement_dict)
print str
print
print new_str
which produces...
structcn = 150.0
metfrac0 = 0.85
structcn = 999.0 = 150.0
metfrac0 = 0.85 = 0.85
thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: XML header with lxml
Jabba Laci, 04.04.2011 18:54:
I want to construct an XML file with lxml but I don't find how to add
the '' header.
This is not required. According to the XML spec, the default is:
So, unless you diverge from these values, you do not need an XML
declaration in your file. If you want to output the declaration anyway, you
can simply prepend it to the string or write it into the output file before
serialising.
from lxml import etree as ET
html = ET.Element("html")
print ET.tostring(html)
simply prints
I imagine you are aware that HTML isn't XML.
Stefan
--
http://mail.python.org/mailman/listinfo/python-list
Re: XML header with lxml
Chroma Key, 04.04.2011 21:49:
On 2011-04-04 18:54:40 +0200, Jabba Laci said:
I want to construct an XML file with lxml but I don't find how to add
the '' header.
from lxml import etree as ET
html = ET.Element("html")
print ET.tostring(html)
Add the "xml_declaration=True" as an argument of etree.tostring().
Ah, yes. That's the right way to do it (if you need it).
Stefan
--
http://mail.python.org/mailman/listinfo/python-list
Re: replace regex in file using a dictionary
2011/4/5 Martin De Kauwe :
> Hi,
>
> So i want to replace multiple lines in a text file and I have reasoned
> the best way to do this is with a dictionary. I have simplified my
> example and broadly I get what I want however I am now printing my
> replacement string and part of the original expression. I am guessing
> that I will need to use a more complicated expression than I am
> currently am to achieve what I want?
>
> my aim is to match the expression before the "=" and replace the
> string after the equals...
>
> My failed example...
> [...]
>
> thanks.
> --
>
Hi,
I guess, you would have to split the line and extract the substrings
before and after "=" and replace only the value part.
Or you can use regular expression replace with a replacement function,
e.g. like the following sample.
The re pattern would probably need some tweaking to match the variable
names, the values and the whitespace around "="; and, of course, it
shouldn't match anything unwanted in the original text ...
hth,
vbr
import re
orig_str = """structcn = 150.0
metfrac0 = 0.85
unknown7 = 38.2
"""
replacement_dict = {"structcn": "999.0", "metfrac0": "0.85"}
def repl_fn(m):
return m.group(1) + m.group(2) + replacement_dict.get(m.group(1),
m.group(3))
new_str = re.sub(r"([\w\d_]+?)( = )([\d.-]+)", repl_fn, orig_str)
print new_str
--
http://mail.python.org/mailman/listinfo/python-list
3D subfigures in matplotlib?
How can I make maybe 3D subfigures with Python and matplotlib?
I found the following example (http://matplotlib.sourceforge.net/
examples/mplot3d/subplot3d_demo.html), but it doesn't work.
---
from mpl_toolkits.mplot3d.axes3d import Axes3D
import matplotlib.pyplot as plt
# imports specific to the plots in this example
import numpy as np
from matplotlib import cm
from mpl_toolkits.mplot3d.axes3d import get_test_data
fig = plt.figure(figsize=(9.5,5.0))
# First subplot
ax = fig.add_subplot(1, 2, 1, projection='3d')
X = np.arange(-5, 5, 0.25)
Y = np.arange(-5, 5, 0.25)
X, Y = np.meshgrid(X, Y)
R = np.sqrt(X**2 + Y**2)
Z = np.sin(R)
surf = ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.jet,
linewidth=0, antialiased=False)
ax.set_zlim3d(-1.01, 1.01)
fig.colorbar(surf, shrink=0.5, aspect=10)
# Second subplot
ax = fig.add_subplot(1, 2, 2, projection='3d')
X, Y, Z = get_test_data(0.05)
ax.plot_wireframe(X, Y, Z, rstride=10, cstride=10)
plt.show()
The error message is:
>>> ax = fig.add_subplot(1, 2, 1, projection='3d')
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/pymodules/python2.6/matplotlib/figure.py", line 677,
in add_subplot
projection_class = get_projection_class(projection)
File "/usr/lib/pymodules/python2.6/matplotlib/projections/
__init__.py", line 61, in get_projection_class
raise ValueError("Unknown projection '%s'" % projection)
ValueError: Unknown projection '3d'
>>>
I am using Python 2.6.6., and matplotlib 0.99.3.
How can I make several subfigures in 3D?
--
http://mail.python.org/mailman/listinfo/python-list
Re: replace regex in file using a dictionary
How about creating a new string instead of replacing the original.
def replace_keys(text, replacements_dict):
text_list = text.split('\n')
new_text = ''
for line in text_list:
key=line.split('=')[0]
if key.strip() in replacements_dict:
new_text = new_text + key + '=' + replacements_dict[key.strip()]
+ "\n"
else:
new_text = new_text + line
return new_text
On 5 April 2011 09:32, Vlastimil Brom wrote:
> 2011/4/5 Martin De Kauwe :
> > Hi,
> >
> > So i want to replace multiple lines in a text file and I have reasoned
> > the best way to do this is with a dictionary. I have simplified my
> > example and broadly I get what I want however I am now printing my
> > replacement string and part of the original expression. I am guessing
> > that I will need to use a more complicated expression than I am
> > currently am to achieve what I want?
> >
> > my aim is to match the expression before the "=" and replace the
> > string after the equals...
> >
> > My failed example...
> > [...]
> >
> > thanks.
> > --
> >
>
> Hi,
> I guess, you would have to split the line and extract the substrings
> before and after "=" and replace only the value part.
> Or you can use regular expression replace with a replacement function,
> e.g. like the following sample.
> The re pattern would probably need some tweaking to match the variable
> names, the values and the whitespace around "="; and, of course, it
> shouldn't match anything unwanted in the original text ...
>
> hth,
> vbr
>
>
> import re
>
> orig_str = """structcn = 150.0
> metfrac0 = 0.85
> unknown7 = 38.2
> """
>
> replacement_dict = {"structcn": "999.0", "metfrac0": "0.85"}
>
> def repl_fn(m):
>return m.group(1) + m.group(2) + replacement_dict.get(m.group(1),
> m.group(3))
>
> new_str = re.sub(r"([\w\d_]+?)( = )([\d.-]+)", repl_fn, orig_str)
>
> print new_str
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Python benefits over Cobra
I just came across the Cobra language, which appears to be heavily influenced by Python (and other languages). The pitch sounds great. It's supposed to have: 1. Quick, expressive coding 2. Fast execution 3. Static and dynamic binding 4. Language level support for quality http://cobra-language.com/docs/why/ http://cobra-language.com/docs/python/ I was wondering what advantages Python has over Cobra. I know it's probably a difficult question to answer and depends on specific requirements. All I can think of is: * Maturity of language o Robust and tested. o availability of modules (standard and built-in). o large community support (commercial and non-commercial). * No dependence of .NET/Mono o I don't know if this is an pro or con as I don't know .NET. Presumably the maturity argument would be less significant over time. I'm not sure about the .NET/Mono framework, whether that is good or bad. Sounds good in some situations at least. Any other arguments where Python has benefits over Cobra ?? Cheers, Brendan. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python benefits over Cobra
On 05-Apr-11 06:22 AM, Brendan Simon (eTRIX) wrote: I just came across the Cobra language, which appears to be heavily influenced by Python (and other languages). The pitch sounds great. It's supposed to have: 1. Quick, expressive coding 2. Fast execution 3. Static and dynamic binding 4. Language level support for quality http://cobra-language.com/docs/why/ http://cobra-language.com/docs/python/ I was wondering what advantages Python has over Cobra. I know it's probably a difficult question to answer and depends on specific requirements. All I can think of is: * Maturity of language o Robust and tested. o availability of modules (standard and built-in). o large community support (commercial and non-commercial). * No dependence of .NET/Mono o I don't know if this is an pro or con as I don't know .NET. Presumably the maturity argument would be less significant over time. I'm not sure about the .NET/Mono framework, whether that is good or bad. Sounds good in some situations at least. Any other arguments where Python has benefits over Cobra ?? Cheers, Brendan. Two questions: 1. Is Cobra Open Source? 2. The blog ended on October, did he run out of steam? I liked the '.', in place of '.self', but that's been rejected for Python. Colin W. -- http://mail.python.org/mailman/listinfo/python-list
is python 3 better than python 2?
what are the advantages? if it wasn't for python 3 breaking backwards compatibility would it be the better choice? -- http://mail.python.org/mailman/listinfo/python-list
python fighting game?(like street fighter)
are there any good fighting games in the style of street fighter or guilty gear? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python benefits over Cobra
Cobra seems interessant, open-source, but the dependance on Mono and .Net annoy me a bit. Otherwise, cobra have good ideas, a syntax similar to python. One thing i really love is the "How-To" and the "Samples" pages on it's website, i think it's a very good thing for beginners. FELD Boris 2011/4/5 Colin J. Williams : > On 05-Apr-11 06:22 AM, Brendan Simon (eTRIX) wrote: >> >> I just came across the Cobra language, which appears to be heavily >> influenced by Python (and other languages). The pitch sounds great. It's >> supposed to have: >> >> 1. Quick, expressive coding >> 2. Fast execution >> 3. Static and dynamic binding >> 4. Language level support for quality >> >> >> http://cobra-language.com/docs/why/ >> >> http://cobra-language.com/docs/python/ >> >> I was wondering what advantages Python has over Cobra. I know it's >> probably a difficult question to answer and depends on specific >> requirements. All I can think of is: >> >> * Maturity of language >> o Robust and tested. >> o availability of modules (standard and built-in). >> o large community support (commercial and non-commercial). >> * No dependence of .NET/Mono >> o I don't know if this is an pro or con as I don't know .NET. >> >> >> Presumably the maturity argument would be less significant over time. >> >> I'm not sure about the .NET/Mono framework, whether that is good or bad. >> Sounds good in some situations at least. >> >> Any other arguments where Python has benefits over Cobra ?? >> >> Cheers, Brendan. >> > Two questions: > 1. Is Cobra Open Source? > 2. The blog ended on October, did he run out of steam? > > I liked the '.', in place of '.self', but that's been rejected for Python. > > Colin W. > > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Suggest some exercises on Python
Hi all, I'm a new bie. I have just started learning Python (3.0), finished with official tutorial. I would like to have your opinion on some 1. Good books (for an intermediate in programming) with lot's of exercise i can try it out. For example when i started with Perl i was going through a book 'Learning Perl' and it contained lots of exercises which made me confident. 2. Or any link or site which suggests some exercises to do in Python. When i was looking for some similar threads in some sites, they propose some sites which gives some exercises and puzzles to solve in python. But i'm feeling it's too much complicated for my capability. Cheers Jins thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
>> what is the character limit on a one liner :P. > > For PEP 8 compliance, 80 characters. :-) Yeah, but we don't live in the 80's or 90's anymore and our screens can support xterms (or let alone IDE widows) much wider than 80 characters. I'm using 140 for python these days. Seriously, who would want to limit him/herself to 80 characters in 2011? Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: proposal to allow to set the delimiter in str.format to something other than curly bracket
On 2011-04-04, Alia Khouri wrote: > Nice! I didn't think of that. I guess I could get some > additional performance by taking the re.compile step out of the > function. Thanks for the tip! I recommend "The Pragmatic Programmer," (Hunt, Thomas) ISBN 978-0201616224 and "The Practice of Programming," (Kernighan, Pike) ISBN 978-0201615869 for more practical tips like this one. As a mostly self-taught programmer, both books helped me immensely. If you're a seasoned pro with a CS degree, you might not find them quite as interesting, though. The second book, especially, does not cover issues relating to large scale software development. As for factoring out re.compile, I believe they are cached by the re module, so you would save the cost of retrieving the cached regex, but not the cost of building it. -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: running Python2 Python3 parallel concurrent
rusi wrote: Not sure if this is relevant... python-mode (emacs mode for python development) is broken for python 3 I submitted a 1-line patch which makes python-mode work for 2.x and 3.x Its relevant to Mac OSX, but not so much for what I'm doing in Linux, here. On the other hand, I can't see (at the moment) why the technique I'm showing here would not work on the Mac. Although, I have not compiled sources on my Mac yet. That is for another day. In fact, I really think I'm about ready to scrap the Mac OSX and just run BLAG linux on it... I have a Mac Mini and I'm fed up with proprietary software (OS and other). Well, if you read PEP 394 you will get a really good idea of what is wrong with windows way down under its lame covers. But that is also another story for another day. kind regards, m harris -- http://mail.python.org/mailman/listinfo/python-list
Re: is python 3 better than python 2?
neil wrote: > what are the advantages? if it wasn't for python 3 breaking backwards > compatibility would it be the better choice? IMHO the killer app for Python 3 is in more reasonable support for foreign character sets (no matter where your are, at least one out of the hundred-odd Unicode character sets is going to be foreign,) and generally internationalized data processing. Mel. -- http://mail.python.org/mailman/listinfo/python-list
How to program in Python to run system commands in 1000s of servers
Here is my problem: Want to program in python to run sysadmin commands across 1000s of servers and gather the result in one place. Many times the commands need to be run as root. We cannot use ssh as root remote connectivity as well. What are the different ways of programming in python to achieve this? -- http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
On Tue, 05 Apr 2011 15:38:28 +0200, Daniel Fetchinson wrote: >>> what is the character limit on a one liner :P. >> >> For PEP 8 compliance, 80 characters. :-) > > Yeah, but we don't live in the 80's or 90's anymore and our screens can > support xterms (or let alone IDE widows) much wider than 80 characters. > I'm using 140 for python these days. Seriously, who would want to limit > him/herself to 80 characters in 2011? Seriously, or is that a rhetorical question? People who like to have two source files side-by-side on a standard sized monitor, or three on a wide-screen monitor. People who need to read or edit source code on a virtual terminal rather than in a GUI console app. People who print out source code to read later. People with poor vision who need to use a significantly larger sized characters, and therefore can't fit as many on a line. People who might want to email code snippets without having them inconveniently wrapped by the mail client. People who don't like reading really long, overly complex, lines, and prefer to keep lines short for readability. And most importantly... people who want to have their code accepted into the Python standard library. Personally, I find that the discipline of keeping to 80 characters is good for me. It reduces the temptation of writing obfuscated Python one- liners when two lines would be better. The *only* time it is a burden is when I write doc strings, and even then, only a small one. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Suggest some exercises on Python
On Tue, 2011-04-05 at 18:43 +0530, Jins Thomas wrote: > Hi all, > > I'm a new bie. I have just started learning Python (3.0), finished > with official tutorial. I would like to have your opinion on some > > 1. Good books (for an intermediate in programming) with lot's of > exercise i can try it out. For example when i started with Perl i was > going through a book 'Learning Perl' and it contained lots of > exercises which made me confident. > > 2. Or any link or site which suggests some exercises to do in Python. > When i was looking for some similar threads in some sites, they > propose some sites which gives some exercises and puzzles to solve in > python. But i'm feeling it's too much complicated for my capability. > > > Cheers > Jins thomas i suggest you try http://www.pythonchallenge.com/ its pretty fun and very hard as it goes on. this book has you write a game with each chapter, http://inventwithpython.com/chapters/ the first 4 chapters you can probably skip. -- http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
On Wed, Apr 6, 2011 at 1:07 AM, Steven D'Aprano
wrote:
> On Tue, 05 Apr 2011 15:38:28 +0200, Daniel Fetchinson wrote:
>
>> Seriously, who would want to limit
>> him/herself to 80 characters in 2011?
>
> Seriously, or is that a rhetorical question?
>
> People who like to have two source files side-by-side on a standard
> sized monitor, or three on a wide-screen monitor.
>
> And most importantly... people who want to have their code accepted into
> the Python standard library.
Is that 80 including indentation, or excluding? And if including, does
that put a hard limit of 20 indentation levels for standard library
code?
Only partly tongue-in-cheek. I have code that quite legitimately has
gone to ten tabs in, and stayed there, and has on occasion gone as far
as 12-16. Generally I just scroll my display horizontally and ignore
the preceding tab levels. And sometimes I cheat a bit, in C or Pike,
and don't indent at some level; if it's a loop (or more likely a
switch) that surrounds the entire function, I might do:
void functionname(parameters) {while (cond)
{
//code at one indent, not two
}}
Can't do that in Python - for better or for worse.
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
On Tue, 2011-04-05 at 15:38 +0200, Daniel Fetchinson wrote: > Yeah, but we don't live in the 80's or 90's anymore and our screens > can support xterms (or let alone IDE widows) much wider than 80 > characters. I'm using 140 for python these days. Seriously, who would > want to limit him/herself to 80 characters in 2011? I'd rather have two files open with 80 columns in them than a single file with 160 columns and have to switch between files. Tim Wintle -- http://mail.python.org/mailman/listinfo/python-list
Re: replace regex in file using a dictionary
On Apr 5, 3:59 am, Martin De Kauwe wrote:
> Hi,
>
> So i want to replace multiple lines in a text file and I have reasoned
> the best way to do this is with a dictionary. I have simplified my
> example and broadly I get what I want however I am now printing my
> replacement string and part of the original expression. I am guessing
> that I will need to use a more complicated expression than I am
> currently am to achieve what I want?
>
> my aim is to match the expression before the "=" and replace the
> string after the equals...
>
> My failed example...
>
> def replace_keys(text, replacements_dict):
> for key, var in replacements_dict.iteritems():
> replacement = "%s = %s" % (key, var)
> text = text.replace(key, replacement)
> return text
>
> # example of two lines of the file
> str = \
> """structcn = 150.0
> metfrac0 = 0.85
> """
>
> # replacements
> replacement_dict = {"structcn": "999.0", "metfrac0": "0.85"}
> new_str = replace_keys(str, replacement_dict)
>
> print str
> print
> print new_str
>
> which produces...
>
> structcn = 150.0
> metfrac0 = 0.85
>
> structcn = 999.0 = 150.0
> metfrac0 = 0.85 = 0.85
>
> thanks.
If the structure is very regular you can use something like this:
def replace_keys(text, replacements_dict):
lines=text.splitlines()
for i, row in enumerate(lines):
key, sep, val = row.split()
lines[i]=" ".join(
(key, sep, replacement_dict.get(key, val)))
return '\n'.join(lines)+'\n'
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing, shared memory vs. pickled copies
On 4/4/11 7:05 PM, Robert Kern wrote: On 4/4/11 3:20 PM, John Ladasky wrote: However, at least in Python 2.7, multiprocessing seems to have a C extension module defining the Connection objects. Unfortunately, it looks like this C extension just imports the regular pickler that is not aware of these custom types. That's why you get this error. I believe this is a bug in Python. So what did you try, and what output did you get? What version of Python are you using? I think that my understanding of multiprocessing needs to evolve beyond the use of Pool, but I'm not sure yet. This post suggests as much. http://mail.scipy.org/pipermail/scipy-user/2009-February/019696.html Maybe. If the __reduce_ex__() method is implemented properly (and multiprocessing bugs aren't getting in the way), you ought to be able to pass them to a Pool just fine. You just need to make sure that the shared arrays are allocated before the Pool is started. And this only works on UNIX machines. The shared memory objects that shmarray uses can only be inherited. I believe that's what Sturla was getting at. I'll take that back a little bit. Since the underlying shared memory types can only be shared by inheritance, the ForkingPickler is only used for the arguments passed to Process(), and not used for things sent through Queues (which Pool uses). Since Pool cannot guarantee that the data exists before the Pool starts its subprocesses, it must use the general mechanism. So in short, if you pass the shmarrays as arguments to the target function in Process(), it should work fine. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: is python 3 better than python 2?
On 4/5/2011 8:42 AM, neil wrote: what are the advantages? Py3 complete many transitions begun in Py2. In some cases, that means deleting old obsolete stuff, such as old-style classes. if it wasn't for python 3 breaking backwards compatibility would it be the better choice? Assuming equal library availability, absolutely. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: is python 3 better than python 2?
On Apr 5, 6:42 am, neil wrote: > what are the advantages? if it wasn't for python 3 breaking backwards > compatibility would it be the better choice? I adopted Py3 because of support of OrderedDict and many new features. Since mine was a new project using no existing libraries, it made sense. -- Gnarlie -- http://mail.python.org/mailman/listinfo/python-list
Re: How to program in Python to run system commands in 1000s of servers
On Tue, Apr 5, 2011 at 7:51 AM, Babu wrote: > > Here is my problem: Want to program in python to run sysadmin > commands across 1000s of servers and gather the result in one place. > Many times the commands need to be run as root. We cannot use ssh as > root remote connectivity as well. What are the different ways of > programming in python to achieve this? There are a bajillion ways to do it badly, but SSH sounds like the right tool for the job here. You really don't want your remote admin system compromised, and fabric makes this kind of thing really much less painful. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing, shared memory vs. pickled copies
Hi Philip,
Thanks for the reply.
On Apr 4, 4:34 pm, Philip Semanchuk wrote:
> So if you're going to use multiprocessing, you're going to use pickle, and you
> need pickleable objects.
OK, that's good to know.
> > Pickling is still a pretty
> > vague progress to me, but I can see that you have to write custom
> > __reduce__ and __setstate__ methods for your objects.
>
> Well, that's only if one's objects don't support pickle by default. A lot of
> classes do without any need for custom __reduce__ and __setstate__ methods.
> Since you're apparently not too familiar with pickle, I don't want you to get
> the false impression that it's a lot of trouble. I've used pickle a number of
> times and never had to write custom methods for it.
I used __reduce__ and __setstate__ once before with success, but I
just hacked at code I found on the Net, without fully understanding
it.
All right, since my last post, I've made an attempt to understand a
bit more about pickle. In doing so, I'm getting into the guts of the
Python class model, in a way that I did not expect. I believe that I
need to understand the following:
1) What kinds of objects have a __dict__,
2) Why the default pickling methods do not simply look for a __dict__
and serialize its contents, if it is present, and
3) Why objects can exist that do not support pickle by default, and
cannot be correctly pickled without custom __reduce__ and __setstate__
methods.
I can only furnish a (possibly incomplete) answer to question #1.
User subclasses apparently always have a __dict__. Here:
>>> class Foo:
... pass # It doesn't get any simpler
...
>>> x = Foo()
>>> x
<__builtin__.Foo instance at 0x9b916cc>
>>> x.__dict__
{}
>>> x.spam = "eggs"
>>> x.__dict__
{'spam': 'eggs'}
>>> setattr(x, "parrot", "dead")
<__builtin__.Bar instance at 0x9b916cc>
>>> dir(x)
['__doc__', '__module__', 'parrot', 'spam']
>>> x.__dict__
{'parrot': 'dead', 'spam': 'eggs'}
>>> x.__dict__ = {"a":"b", "c":"d"}
>>> dir(x)
['__doc__', '__module__', 'a', 'c']
>>> x.__dict__
{'a': 'b', 'c': 'd'}
Aside: this finding makes me wonder what exactly dir() is showing me
about an object (all objects in the namespace, including methods, but
not __dict__ itself?), and why it is distinct from __dict__.
In contrast, it appears that BUILT-IN classes (numbers, lists,
dictionaries, etc.) do not have a __dict__. And that you can't force
a __dict__ into built-in objects, no matter how you may try.
>>> n = 1
>>> dir(n)
['__abs__', '__add__', '__and__', '__class__', '__cmp__',
'__coerce__',
'__delattr__', '__div__', '__divmod__', '__doc__', '__float__',
'__floordiv__',
'__format__', '__getattribute__', '__getnewargs__', '__hash__',
'__hex__',
'__index__', '__init__', '__int__', '__invert__', '__long__',
'__lshift__',
'__mod__', '__mul__', '__neg__', '__new__', '__nonzero__',
'__oct__', '__or__',
'__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__',
'__rdivmod__',
'__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__',
'__rlshift__',
'__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__',
'__rshift__',
'__rsub__', '__rtruediv__', '__rxor__', '__setattr__',
'__sizeof__', '__str__',
'__sub__', '__subclasshook__', '__truediv__', '__trunc__',
'__xor__', 'conjugate',
'denominator', 'imag', 'numerator', 'real']
>>> n.__dict__
File "", line 1, in
''' : 'int' object has no
attribute '__dict__' '''
>>> n.__dict__ = {"a":"b", "c":"d"}
File "", line 1, in
''' : 'int' object has no
attribute '__dict__' '''
>>> setattr(n, "__dict__", {"a":"b", "c":"d"})
File "", line 1, in
''' : 'int' object has no
attribute '__dict__' '''
This leads straight into my second question. I THINK, without knowing
for sure, that most user classes would pickle correctly by simply
iterating through __dict__. So, why isn't this the default behavior
for Python? Was the assumption that programmers would only want to
pickle built-in classes? Can anyone show me an example of a common
object where my suggested approach would fail?
> What might be easier than fooling around with boxes of sharp knives is to
> convert
> your ndarray objects
... with their attendant meta-data, of course...
> to Python lists. Lists are pickle-friendly and easy to turn
> back into ndarray objects once they've crossed the pickle boundary.
That's a cute trick. I may try it, but I would rather do it the way
that Python recommends. I always prefer to understand what I'm doing,
and why.
> Python will only get better at working with multiple cores/CPUs, but there's
> plenty of room for improvement on the status quo.
Thank you for agreeing with me. I'm not formally trained in computer
science, and there was always the chance that my opinion as a non-
professional might come from simply not understanding the issues well
enough. But I chose Python as a pro
Re: Suggest some exercises on Python
On 4/5/2011 9:13 AM, Jins Thomas wrote: I'm a new bie. I have just started learning Python (3.0), Please download, install, and use 3.2. finished with official tutorial. > I would like to have your opinion on some 1. Good books (for an intermediate in programming) with lot's of ... 2. Or any link or site which suggests some exercises to do in Python. ... One thing I did was lots of experiments with variations on the example given in the doc, and creation of examples that are not. After 14 years, I still do that with a new-to-me module. I nearly always use the IDLE interpreter window (for one-liners) or editor windows. That said, the problem with such requests is that the proper answer depends what your are interested in. Suppose you are interested in numerical computing. Pick any book numerical algorithms or analysis and pick some examples with answers given. Most likely, the book uses some combination of natural language, pseudocode, and Fortran or C. Rewrite in Python, testing with the inputs and outputs given (keeping in mind that floating point answers will vary in the last digits). Suppose you are more interested in interacting with the web. Write something to fetch a page (from python.org, say) and do something with it. Try filling out the search form at bugs.python.org and extracting the data table from the response page. Etc. Suppose you are even move interested in xyz.> -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Suggest some exercises on Python
On Tue, 05 Apr 2011 13:18:19 -0400, Terry Reedy wrote: > On 4/5/2011 9:13 AM, Jins Thomas wrote: > >> I'm a new bie. I have just started learning Python (3.0), > > Please download, install, and use 3.2. To elaborate further: Python 3.0 is not supported, and is buggy and slow. You should avoid it. Python 3.1 is the first production-ready version of Python 3, and 3.2 is the current version. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Light weight IDE like pywin for os x 10.6
python wrote: > I have looked a while for this answer. Sorry if it right before me. > > I have move from Windows to os x. The thing I miss most is pywin. > I know you can purchase or download full IDE's for python or even use > Eclipse. I really liked the ability to switch from interactive > interpreter to editor easily. The autocompletion is very helpful so > idle does not quite cut it for me. How about eric (http://eric-ide.python-projects.org)? Detlev -- Detlev Offenbach det...@die-offenbachs.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Suggest some exercises on Python
On Tue, Apr 5, 2011 at 6:13 AM, Jins Thomas wrote: > Hi all, > > I'm a new bie. I have just started learning Python (3.0), finished with > official tutorial. I would like to have your opinion on some > > 1. Good books (for an intermediate in programming) with lot's of exercise i > can try it out. For example when i started with Perl i was going through a > book 'Learning Perl' and it contained lots of exercises which made me > confident. > > 2. Or any link or site which suggests some exercises to do in Python. When i > was looking for some similar threads in some sites, they propose some sites > which gives some exercises and puzzles to solve in python. But i'm feeling > it's too much complicated for my capability. If you're mathematically inclined, try Project Euler. It isn't python-specific, but it is a lot of fun. http://projecteuler.net/ Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
On Wed, 06 Apr 2011 01:19:06 +1000, Chris Angelico wrote: > On Wed, Apr 6, 2011 at 1:07 AM, Steven D'Aprano > wrote: >> On Tue, 05 Apr 2011 15:38:28 +0200, Daniel Fetchinson wrote: >> >>> Seriously, who would want to limit >>> him/herself to 80 characters in 2011? >> >> Seriously, or is that a rhetorical question? >> >> People who like to have two source files side-by-side on a standard >> sized monitor, or three on a wide-screen monitor. >> >> And most importantly... people who want to have their code accepted >> into the Python standard library. > > Is that 80 including indentation, or excluding? And if including, does > that put a hard limit of 20 indentation levels for standard library > code? Including. As for the hard limit, pretty much. > Only partly tongue-in-cheek. I have code that quite legitimately has > gone to ten tabs in, and stayed there, "Legitimately"? I very much doubt it. (Only half joking.) > and has on occasion gone as far as 12-16. I would consider anything more than four indents a code smell. That is, four is unexceptional; five would make me look over the code to see if it could be refactored; six would make me look at it *carefully*; eight would have me questioning my own sanity. I wouldn't give a hard limit beyond which I would "never" go beyond, but I find it difficult to imagine what drugs I'd need to be on to go beyond eight. *wink* -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
what is the character limit on a one liner :P. >>> >>> For PEP 8 compliance, 80 characters. :-) >> >> Yeah, but we don't live in the 80's or 90's anymore and our screens can >> support xterms (or let alone IDE widows) much wider than 80 characters. >> I'm using 140 for python these days. Seriously, who would want to limit >> him/herself to 80 characters in 2011? > > Seriously, or is that a rhetorical question? Rhetorical :) What it intended to demonstrate is that it is possible to rationalize ancient habits which have their origins in particular circumstances of those ancient times but actually nothing necessitates them today other than the momentum built around them during their lifetime (which is a pretty strong argument for them by the way). Cheers, Daniel > People who like to have two source files side-by-side on a standard > sized monitor, or three on a wide-screen monitor. > > People who need to read or edit source code on a virtual terminal rather > than in a GUI console app. > > People who print out source code to read later. > > People with poor vision who need to use a significantly larger sized > characters, and therefore can't fit as many on a line. > > People who might want to email code snippets without having them > inconveniently wrapped by the mail client. > > People who don't like reading really long, overly complex, lines, and > prefer to keep lines short for readability. > > And most importantly... people who want to have their code accepted into > the Python standard library. > > > Personally, I find that the discipline of keeping to 80 characters is > good for me. It reduces the temptation of writing obfuscated Python one- > liners when two lines would be better. The *only* time it is a burden is > when I write doc strings, and even then, only a small one. > > > > -- > Steven > -- > http://mail.python.org/mailman/listinfo/python-list > -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing, shared memory vs. pickled copies
On Apr 5, 2011, at 12:58 PM, John Ladasky wrote: > Hi Philip, > > Thanks for the reply. > > On Apr 4, 4:34 pm, Philip Semanchuk wrote: >> So if you're going to use multiprocessing, you're going to use pickle, and >> you >> need pickleable objects. > > OK, that's good to know. But as Dan Stromberg pointed out, there are some pickle-free ways to communicate between processes using multiprocessing. > This leads straight into my second question. I THINK, without knowing > for sure, that most user classes would pickle correctly by simply > iterating through __dict__. So, why isn't this the default behavior > for Python? Was the assumption that programmers would only want to > pickle built-in classes? One can pickle user-defined classes: >>> class Foo(object): ... pass ... >>> import pickle >>> foo_instance = Foo() >>> pickle.dumps(foo_instance) 'ccopy_reg\n_reconstructor\np0\n(c__main__\nFoo\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n.' And as Robert Kern pointed out, numpy arrays are also pickle-able. >>> import numpy >>> pickle.dumps(numpy.zeros(3)) "cnumpy.core.multiarray\n_reconstruct\np0\n(cnumpy\nndarray\np1\n(I0\ntp2\nS'b'\np3\ntp4\nRp5\n(I1\n(I3\ntp6\ncnumpy\ndtype\np7\n(S'f8'\np8\nI0\nI1\ntp9\nRp10\n(I3\nS'<'\np11\nNNNI-1\nI-1\nI0\ntp12\nbI00\nS'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'\np13\ntp14\nb." As a side note, you should always use "new style" classes, particularly since you're exploring the details of Python class construction. "New" is a bit a of misnomer now, as "new" style classes were introduced in Python 2.2. They have been the status quo in Python 2.x for a while now and are the only choice in Python 3.x. Subclassing object gives you a new style class: class Foo(object): Not subclassing object (as you did in your example) gives you an old style class: class Foo: Cheers Philip -- http://mail.python.org/mailman/listinfo/python-list
Indentation levels and line lengths (was Re: Fun python 3.2 one-liner)
On Wed, Apr 6, 2011 at 3:48 AM, Steven D'Aprano
wrote:
>> and has on occasion gone as far as 12-16.
>
> I would consider anything more than four indents a code smell. That is,
> four is unexceptional; five would make me look over the code to see if it
> could be refactored; six would make me look at it *carefully*; eight
> would have me questioning my own sanity. I wouldn't give a hard limit
> beyond which I would "never" go beyond, but I find it difficult to
> imagine what drugs I'd need to be on to go beyond eight.
>
> *wink*
Chocolate. Chocolate and heavy C code work. The alternative is to
break everything out into separate functions, which inevitably results
in heaps of unnecessary parameter-passing; granted, a C compiler will
often optimize most of it away, but it's still annoying to have to
pass a whole lot of reference parameters to a function that's only
ever called from one place.
As an example, think of a typical GUI message loop (in OS/2 or Windows):
int WndProc(HWND hwnd,int msg,int param1,int param2)
{
switch (msg)
{
case WM_PAINT:
{
if (have_message)
{
display_message();
y+=msg_height;
}
}
//etc, etc, etc
default: return 0;
}
}
That's four levels of indentation right there. Now imagine that the
whole kaboodle is inside a class definition and that's a fifth, and
then it only takes a small amount of extra code complexity to hit
eight. I agree that it's a code smell, though. For something to get to
the 10 or 12 that I mentioned above, it really has to be something
with a whole lot of relatively insignificant indentations early on, so
the entire "useful logic" is contained in levels 6-10 or something;
and the 16 was probably a little artificial, in that I *could* have
refactored that function but just never got around to it (which means
it's pushing the limits of "legitimately"). But I know I've had C code
get past 10 without there being any logical reason to break some of
the code out.
The other time I have miles of indentation is when I'm working in an
unfamiliar language, and I'm not comfortable with its variable scoping
rules (hello there, PHP). That's not what I'd call "legitimate"
indentation, but I sure am glad there's no hard limit on what's
accepted.
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Suggest some exercises on Python
On 05/04/2011 18:24, Steven D'Aprano wrote: On Tue, 05 Apr 2011 13:18:19 -0400, Terry Reedy wrote: On 4/5/2011 9:13 AM, Jins Thomas wrote: I'm a new bie. I have just started learning Python (3.0), Please download, install, and use 3.2. To elaborate further: Python 3.0 is not supported, and is buggy and slow. You should avoid it. Python 3.1 is the first production-ready version of Python 3, and 3.2 is the current version. You can always go back to 2.7 when you realise that something you want to use is not yet fully supported, in my case matplotlib. Mark L. -- http://mail.python.org/mailman/listinfo/python-list
Re: is python 3 better than python 2?
On 4/5/2011 7:46 AM, Mel wrote: neil wrote: what are the advantages? if it wasn't for python 3 breaking backwards compatibility would it be the better choice? IMHO the killer app for Python 3 is in more reasonable support for foreign character sets (no matter where your are, at least one out of the hundred-odd Unicode character sets is going to be foreign,) and generally internationalized data processing. Well, actually Unicode support went in back around Python 2.4. In 3.x, ASCII strings went away, but that was more of a removal. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
a better way to invert a list?
Greetings, I've been playing around (in Python 3.1) with permutations of 0,1,...,n-1, represented by lists, p, of length n, where p[i] = the image of i under the permutation. I wanted to be able to calculate the inverse of such a permutation, and came up with the following succint but not quite readable function: def invert(p): return [ j for (i,j) in sorted(zip(p,range(len(p] I rejected the obvious [p.index(i) for i in range(len(p))] since that seems an inefficient way to sort. Is there a better way to do this? Another question about my code: What is more idiomatic, [ j for (i,j) in ...] or [ x[1] for x in ... ]? I find the former more readable. But it makes bindings for i and doesn't use them - which can't be very efficient. In Haskell or ML, you can use patterns that contain wild cards that play a role in the pattern-matching but don't establish any binding. Can that be done in Python? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: a better way to invert a list?
On Tue, Apr 5, 2011 at 3:17 PM, scattered wrote: > Greetings, > > I've been playing around (in Python 3.1) with permutations of > 0,1,...,n-1, represented by lists, p, of length n, where p[i] = the > image of i under the permutation. I wanted to be able to calculate the > inverse of such a permutation, and came up with the following succint > but not quite readable function: > > def invert(p): > return [ j for (i,j) in sorted(zip(p,range(len(p] > > I rejected the obvious [p.index(i) for i in range(len(p))] since that > seems an inefficient way to sort. Is there a better way to do this? Instead of zipping up range(len(p)), you could use the more Pythonic enumerate: def invert(p): return [i for (i, j) in sorted(enumerate(p), key=operator.itemgetter(1))] The main advantage here is that it will accept an iterator for p, not just a sequence. But it's still O(n log n ) due to the sort. More efficient would be: def invert(p): inverse = [None] * len(p) for (i, j) in enumerate(p): inverse[j] = i return inverse Which is O(n). If that is too verbose, you could also use a dictionary: def invert(p): return dict(map(reversed, enumerate(p))) But the disadvantage here is that if you want to iterate over the result in order, you're back to either having to sort it or doing something ugly like this: q = invert(p) for i in range(len(q)): # Do something with q[i] > Another question about my code: What is more idiomatic, [ j for (i,j) > in ...] or [ x[1] for x in ... ]? I find the former more readable. > But it makes bindings for i and doesn't use them - which can't be very > efficient. I don't know of any reason to prefer one over the other. One convention for unpacking values that aren't used is to name the variable '_'. But this doesn't help efficiency at all; it's just a convention to inform somebody reading the code that the value is ignored. > In Haskell or ML, you can use patterns that contain wild > cards that play a role in the pattern-matching but don't establish any > binding. Can that be done in Python? No, unfortunately. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: replace regex in file using a dictionary
yes thanks both work nicely, I will ponder the suggestions. -- http://mail.python.org/mailman/listinfo/python-list
Re: a better way to invert a list?
On Apr 5, 5:46 pm, Ian Kelly wrote: > On Tue, Apr 5, 2011 at 3:17 PM, scattered wrote: > > Greetings, > > > I've been playing around (in Python 3.1) with permutations of > > 0,1,...,n-1, represented by lists, p, of length n, where p[i] = the > > image of i under the permutation. I wanted to be able to calculate the > > inverse of such a permutation, and came up with the following succint > > but not quite readable function: > > > def invert(p): > > return [ j for (i,j) in sorted(zip(p,range(len(p] > > > I rejected the obvious [p.index(i) for i in range(len(p))] since that > > seems an inefficient way to sort. Is there a better way to do this? > > Instead of zipping up range(len(p)), you could use the more Pythonic > enumerate: > > def invert(p): > return [i for (i, j) in sorted(enumerate(p), key=operator.itemgetter(1))] > Seems a bit kludgy - isn't their any more direct way to sort by the second element in a list of tuples? I gather that this is one area where Python 3 differs from earlier Pythons - but I never had much experience with Python 2 to compare it with. > The main advantage here is that it will accept an iterator for p, not > just a sequence. But it's still O(n log n ) due to the sort. More > efficient would be: > > def invert(p): > inverse = [None] * len(p) > for (i, j) in enumerate(p): > inverse[j] = i > return inverse Elegant. This seems like the best solution, although it isn't as much fun to write as a "one-liner". Thanks > Which is O(n). If that is too verbose, you could also use a dictionary: > > def invert(p): > return dict(map(reversed, enumerate(p))) > > But the disadvantage here is that if you want to iterate over the > result in order, you're back to either having to sort it or doing > something ugly like this: > > q = invert(p) > for i in range(len(q)): > # Do something with q[i] > > > Another question about my code: What is more idiomatic, [ j for (i,j) > > in ...] or [ x[1] for x in ... ]? I find the former more readable. > > But it makes bindings for i and doesn't use them - which can't be very > > efficient. > > I don't know of any reason to prefer one over the other. One > convention for unpacking values that aren't used is to name the > variable '_'. But this doesn't help efficiency at all; it's just a > convention to inform somebody reading the code that the value is > ignored. > > > In Haskell or ML, you can use patterns that contain wild > > cards that play a role in the pattern-matching but don't establish any > > binding. Can that be done in Python? > > No, unfortunately. > > Cheers, > Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: a better way to invert a list?
[Ian Kelly] > Which is O(n). If that is too verbose, you could also use a dictionary: > > def invert(p): > return dict(map(reversed, enumerate(p))) def inv(p): return dict(zip(p, itertools.count())) Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
On Apr 5, 6:38 am, Daniel Fetchinson wrote: > >> what is the character limit on a one liner :P. > > > For PEP 8 compliance, 80 characters. :-) > > Yeah, but we don't live in the 80's or 90's anymore and our screens > can support xterms (or let alone IDE widows) much wider than 80 > characters. I'm using 140 for python these days. Seriously, who would > want to limit him/herself to 80 characters in 2011? I wonder how many people will shorten otherwise clear variable names just to get their lines to fit in 80 characters? Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: 3D subfigures in matplotlib?
On Tuesday, April 5, 2011 6:17:34 AM UTC-4, MATLABdude wrote: > How can I make maybe 3D subfigures with Python and matplotlib? > > I found the following example (http://matplotlib.sourceforge.net/ > examples/mplot3d/subplot3d_demo.html), but it doesn't work. > > [snip] > > I am using Python 2.6.6., and matplotlib 0.99.3. > > How can I make several subfigures in 3D? The demo works fine for me using Matplotlib 1.0.1. Here's a version that should work on your older version of Matplotlib. Instead of adding a subplot with projection='3d', it adds a normal subplot and gets the bounding rectangle. The rectangle is used to configure the Axes3D instance: from mpl_toolkits.mplot3d.axes3d import Axes3D import matplotlib.pyplot as plt # imports specific to the plots in this example import numpy as np from matplotlib import cm from mpl_toolkits.mplot3d.axes3d import get_test_data fig = plt.figure(figsize=(9.5,5.0)) # First subplot rect = fig.add_subplot(1, 2, 1).get_position() ax = Axes3D(fig, rect) X = np.arange(-5, 5, 0.25) Y = np.arange(-5, 5, 0.25) X, Y = np.meshgrid(X, Y) R = np.sqrt(X**2 + Y**2) Z = np.sin(R) surf = ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.jet, linewidth=0, antialiased=False) ax.set_zlim3d(-1.01, 1.01) fig.colorbar(surf, shrink=0.5, aspect=10) # Second subplot rect = fig.add_subplot(1, 2, 2).get_position() ax = Axes3D(fig, rect) X, Y, Z = get_test_data(0.05) ax.plot_wireframe(X, Y, Z, rstride=10, cstride=10) plt.show() -- http://mail.python.org/mailman/listinfo/python-list
Re: How to program in Python to run system commands in 1000s of servers
In article , geremy condra wrote: > On Tue, Apr 5, 2011 at 7:51 AM, Babu wrote: > > > > Here is my problem: Want to program in python to run sysadmin > > commands across 1000s of servers and gather the result in one place. > > Many times the commands need to be run as root. We cannot use ssh as > > root remote connectivity as well. What are the different ways of > > programming in python to achieve this? > > There are a bajillion ways to do it badly, but SSH sounds like the > right tool for the job here. You really don't want your remote admin > system compromised, and fabric makes this kind of thing really much > less painful. Agreed on the fabric (fabfile.org) recommendation. We've been using it for about 6 months. Very handy. I'm not sure how to parse: > We cannot use ssh as root remote connectivity as well. but with 1000's of servers, I really don't see any alternative to ssh, with key authentication. You don't really propose to type passwords at 1000's of machines, do you? As far as fabric goes, it's not perfect, but it's pretty good and if you try to roll your own alternative, you will likely 1) waste a lot of time and money and 2) end up with an inferior solution. -- http://mail.python.org/mailman/listinfo/python-list
Sandboxed Python: memory limits?
Is it possible, and if so is it easy, to limit the amount of memory an embedded Python interpreter is allowed to allocate? I don't want to ulimit/rlimit the process if I don't have to (or rather, I want the process's limit to be high, and the Python limit much lower), but just to force Python to throw MemoryError sooner than it otherwise would (my code can then graciously deal with the exception). Google turned up this thread: http://stackoverflow.com/questions/1760025/limit-python-vm-memory The answers given include resource.setrlimit (which presumably goes straight back to the API, which will affect the whole process), and a simple counter (invasive to the code). But I want something that I can impose from the outside. I have a vague memory of reading somewhere that it's possible to replace the Python memory allocator. This would be an option, if there's no simple way to say "your maximum is now 16MB", but I now can't find it back. Was I hallucinating? Hoping not to reinvent any wheels today! Thanks! Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Standard config file format
I am working on a program to monitor directory file changes and am would like a configuration file. This file would specify email addresses, file and directory locations.. Is there a preferred format to use with python? -- Thanks Vincent Davis -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard config file format
On Tue, Apr 5, 2011 at 7:16 PM, Vincent Davis wrote: > I am working on a program to monitor directory file changes and am would > like a configuration file. This file would specify email addresses, file and > directory locations.. Is there a preferred format to use with python? ConfigParser supports an INI-like format: http://docs.python.org/library/configparser.html JSON is also a popular option: http://docs.python.org/library/json.html Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
Raymond Hettinger writes: > On Apr 5, 6:38 am, Daniel Fetchinson > wrote: > > Yeah, but we don't live in the 80's or 90's anymore and our screens > > can support xterms (or let alone IDE widows) much wider than 80 > > characters. I'm using 140 for python these days. Seriously, who > > would want to limit him/herself to 80 characters in 2011? A broadsheet newspaper allows a whole lot of characters on a single line, yet they still limit the width of each passage of text. The limit on line length is not only for technical reasons, but for readability. > I wonder how many people will shorten otherwise clear > variable names just to get their lines to fit in 80 characters? Not me. When the line is over 78 characters or so, I just break the line at an appropriate point. Line breaks (within bracketing syntax) are far less expensive than uncommunicative variable names. -- \ “I must have a prodigious quantity of mind; it takes me as much | `\ as a week sometimes to make it up.” —Mark Twain, _The Innocents | _o__) Abroad_ | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Python benefits over Cobra
> On 05-Apr-11 06:22 AM, Brendan Simon (eTRIX) wrote: >> >> Any other arguments where Python has benefits over Cobra ?? >> >> Cheers, Brendan. >> > Two questions: > 1. Is Cobra Open Source? > 2. The blog ended on October, did he run out of steam? > > I liked the '.', in place of '.self', but that's been rejected for Python. "Cobra is an open source project under the MIT license." according to the web site. It seems that it mostly, if not all, the work of one person. All code commits seem to be from Charles Esterbrook. It seems the latest release is Oct 2010, but I can see posts in the forum for April 2011, March, Feb, . I too like the '.' in place of self :) However, I don't like _ as line continuation :( Life is tough, eh ?? It also looks like there is work to have it run in a JVM. I presume that means that no .NET/Mono framework is required ?? -- http://mail.python.org/mailman/listinfo/python-list
How to debug c libraries on python?
Hi, everyone: I encouter a problem: I invoke some c libraries in python through the api "CDLL".Possibly, there is sth. wrong in my c libraries. It would appear segment fault when executing some func in c libraries which invoked through "CDLL". How can I debug? I attempted to debug it with pdb, but this is not the problem of python, the fault belongs to c. How can I do? thanks, yuanzheng. -- http://mail.python.org/mailman/listinfo/python-list
Re: is python 3 better than python 2?
On 4/5/2011 4:42 PM, John Nagle wrote: Well, actually Unicode support went in back around Python 2.4. Even earlier, I think, but there were and still are problems with unicode in 2.x. Some were and will only be fixed in 3.x. In 3.x, ASCII strings went away, but that was more of a removal. Yes and no. They were kept slightly modified as bytes, with all of the string methods kept. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: is python 3 better than python 2?
On Tue, Apr 5, 2011 at 10:04 PM, Terry Reedy wrote:
> On 4/5/2011 4:42 PM, John Nagle wrote:
>
> Well, actually Unicode support went in back around Python 2.4.
>>
>
> Even earlier, I think, but there were and still are problems with unicode
> in 2.x. Some were and will only be fixed in 3.x.
>
>
> In 3.x, ASCII strings went away, but that was more of a removal.
>>
>
> Yes and no. They were kept slightly modified as bytes, with all of the
> string methods kept.
>
I suspect not all string methods were kept for the bytes type:
$ /usr/local/cpython-3.2/bin/python
cmd started 2011 Tue Apr 05 11:05:08 PM
Python 3.2 (r32:88445, Feb 20 2011, 16:47:11)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 'a/b/c'.split('/')
['a', 'b', 'c']
>>> b'a/b/c'.split('/')
Traceback (most recent call last):
File "", line 1, in
TypeError: Type str doesn't support the buffer API
>>>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Suggest some exercises on Python
neil harper wrote: I'm a new bie. I have just started learning Python (3.0), finished with official tutorial. I would like to have your opinion on some Opinions around here are pretty much like elbows and knee caps... everybody has them, and yours are really the only ones that matter to you...:) 1. Good books (for an intermediate in programming) with lot's of exercise i can try it out. For example when i started with Perl i was going through a book 'Learning Perl' and it contained lots of exercises which made me confident. http://www.amazon.com/Programming-Python-Complete-Introduction-Language/dp/0321680561/ref=dp_ob_title_bk http://oreilly.com/catalog/9780596158118 http://oreilly.com/catalog/9780596158071 Warning: ... these books are huge. I not only mean they are important, I mean, they are huge. Programming Python 4th ed is over 1650 pages... ! But, they are worth every page, and every penny. kind regards, m harris -- http://mail.python.org/mailman/listinfo/python-list
