Re: shouldn't list comprehension be faster than for loops?

[Steven D'Aprano]

On Sat, 19 Dec 2009 12:28:32 +1100, Ryan Kelly wrote:

>> Anything else being equal, list comprehensions will be the faster
>> becuase they incur fewer name and attribute lookups. It will be the
>> same as the difference between a for loop and a call to map. A list
>> comprehension is basically an enhancement of map.
> 
> Not so.  If you use the "dis" module to peek at the bytecode generated
> for a list comprehension, you'll see it's very similar to that generated
> for an explicit for-loop.  The byte-code for a call to map is very
> different.

"Very similar" and "very different" byte-code mean very little regarding 
speed.


> Basically:  both a for-loop and a list-comp do the looping in python
> bytecode, while a call to map will do the actual looping in C.

This is a classic example of the confirmation fallacy -- if you say that 
for-loops and list-comps are very similar, you need to actually check the 
byte-code of both. You don't. You need to compare the byte-code of all 
three operations, not just two of them, e.g.:

dis.dis(compile("map(f, seq)", '', 'exec'))
dis.dis(compile("[f(x) for x in seq]", '', 'exec'))
dis.dis(compile("L = []\nfor x in seq: L.append(f(x))", '', 'exec'))

But in fact just looking at the byte-code isn't helpful, because it tells 
you nothing about the relative speed of each operation. You need to 
actually time the operations.

>>> from timeit import Timer
>>> t1 = Timer("map(len, 'abcdefgh')", setup='')
>>> t2 = Timer("[len(c) for c in 'abcdefgh']", setup='')
>>> t3 = Timer("""L = []
... for c in 'abcdefgh':
... L.append(len(c))
... """, setup='')
>>>
>>> min(t1.repeat())
3.9076540470123291
>>> min(t2.repeat())
4.5931642055511475
>>> min(t3.repeat())
7.4744069576263428


So, on my PC, with Python 2.5, with this example, a for-loop is about 60% 
slower than a list comp and about 90% slower than map; the list comp is 
about 20% slower than map.

But that only holds for *that* example. Here's another one:


>>> def f(x):
... return 1+2*x+3*x**2
...
>>> values = [1,2,3,4,5,6]
>>> t1 = Timer("map(f, values)", setup='from __main__ import f, values')
>>> t2 = Timer("[f(x) for x in values]", 
... setup='from __main__ import f, values')
>>> 
>>> t3 = Timer("""L = []
... for x in values:
... L.append(f(x))
... """, setup='from __main__ import f, values')
>>>
>>> min(t1.repeat())
7.0339860916137695
>>> min(t2.repeat())
6.8053178787231445
>>> min(t3.repeat())
9.1957418918609619


For this example map and the list comp are nearly the same speed, with 
map slightly slower; but the for-loop is still significantly worse.

Of course, none of these timing tests are terribly significant. The 
actual difference in time is of the order of a millionth of a second per 
call to map compared to the list comp or the for-loop, for these small 
examples. Most of the time you shouldn't care about time differences of 
that magnitude, and write whatever is easiest.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: iterators and views of lists

[Terry Reedy]

On 12/19/2009 12:10 AM, Gregory Ewing wrote:

Terry Reedy wrote:

On the other hand, Python indexes are a form of random access
iterator, the top of the hierarchy.


The term "random access iterator" seems oxymoronic to
me. Iteration is all about operating on things in
sequence. If you're accessing elements arbitrarily,
then you're not iterating.


Python, traditionally, only came with one mutable builtin sequence
type, so the sort function was made a method of that type.


Also it probably operates rather more efficiently
than a generic one would, as it doesn't have to go
through a general mechanism to access elements, but
can take advantage of its knowledge of the internal
structure of a list.


I presume there is no array sort because it is O(n) to copy array to 
list and back again with tolist and fromlist methods.


Anyone needed space-saving of in place in array can write array sort 
with generic quicksort or whatever is appropriate to peculiarities of 
specific data.


tjr



--
http://mail.python.org/mailman/listinfo/python-list

Minor bug in multiprocessing?

[Frank Millman]

Hi all

This is a minor issue, but I thought I would mention it.

I am on Python 2.6.2. I have 'from __future__ import unicode_literals' at 
the top of my programs.

I am experimenting with multiprocessing, and in particular subclassing the 
Process class.

If I create a subclass and pass "name='test'" as one of the arguments, I get 
the following -

Traceback (most recent call last):
  File "F:\junk\multiprocess\mp5.py", line 37, in 
p = Frank(name='test')
  File "F:\junk\multiprocess\mp5.py", line 18, in __init__
self.name = name
  File "C:\Python26\lib\multiprocessing\process.py", line 141, in name
assert isinstance(name, str), 'name must be a string'
AssertionError: name must be a string

If I change the argument to "name=str('test')" there is no error.

For Python 2.x I think the assertion should be "isinstance(name, 
basestring)" to prevent this from happening.

Is this worth reporting, if it has not been reported already?

Thanks

Frank Millman



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Moving from PHP to Python. Is it Possible

[AppRe Godeck]

On Mon, 14 Dec 2009 12:25:16 +0100, Bruno Desthuilliers wrote:

> r0g a écrit :
>> Bruno Desthuilliers wrote:
>>> Sancar Saran a écrit :
>>> (snip)
 My problem is with PHP syntax and performance.  I'm just trying to
 replicate my recepies in python...
>>> Python is not PHP, and trying to write PHP in Python won't buy you
>>> much except pain and frustration.
>> 
>> 
>> I think people are being a little harsh here. Replicating exactly what
>> PHP code does on a micro level i.e. line by line is probably a bad idea
>> but for all we know a lot of the macro level stuff might be fine, or
>> mostly fine i.e. structures, algorithms, classes and functions etc.
> 
> I was talking about trying to replicate PHP's execution model and idioms
> in Python - the "framework" part -, not about application specific
> algos, data structures etc.

Try web2py I think you will surprise yourself with its simplicity and 
speed :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sort the values of a dict

[mattia]

Il Sat, 19 Dec 2009 17:30:27 +1100, Lie Ryan ha scritto:

> On 12/19/2009 9:34 AM, mattia wrote:
>> Can you provide me a much pythonic solution (with comments if possible,
>> so I can actually learn something)?
> 
> If you only need to get i'th element sometimes, sorting the dict is
> fine. Otherwise, you might want to use collections.OrderedDict.

Well, in the python doc OrderedDict is described as a dict that remembers 
the order that keys were first inserted and I don't need this. The fact 
is that I use a structure composed by a date and a list of possible 
solutions found, like
(2009/12/21, (('e', 12, 33), ('r', 4, 11), ('r', 1, 33))) then every 
solution is inserted concurrently in a dictionary. I want to sort the 
solution found to provide, e.g., the first 10 dates found and the best 
result of every date based on the i-th element of the date's list.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How Do I...?

[Victor Subervi]

On Fri, Dec 18, 2009 at 3:03 PM, Tim Chase wrote:

> Victor Subervi wrote:
>
>> How do I...?
>>
>
> Well, you start by reading a book on how to program.  You would then learn
> that what you want (in all likelihood) is a dictionary/map structure for
> dynamically created key/value pairs. Once you have progressed from your
> current apprenticeship and achieved the rank of third-degree journeyman
> programmer, the ways of dynamic variable creation will avail themselves.


Why the arrogance? Why talk down to me?

>
> As stated above, you want a dictionary where your keys are
>
>  'optionNo%d' % i
>
> and your values are "i".  You can also use the more idiomatic
>
>  for i, option in enumerate(ourOptions()):
>...
>
> and skip the manual initialization and incrementation of "i".  Or even more
> succinctly, you could pass the above as a generator to the dict()
> initialization.  But that's a level-2 apprentice bit of code.  Patience
> grasshopper.
>

Why the arrogance?

>
> And as additional weirdness, you don't actually make use of "option" in
> your for-loop...
>

The variable was tied to another set of variables that iterated through the
options. Why be so judgmental? I appreciate your advice, after having
filtered out the attitude.
V
-- 
http://mail.python.org/mailman/listinfo/python-list

numpy performance and random numbers

[Carl Johan Rehn]

Dear friends,

I plan to port a Monte Carlo engine from Matlab to Python. However,
when I timed randn(N1, N2) in Python and compared it with Matlab's
randn, Matlab came out as a clear winner with a speedup of 3-4 times.
This was truly disappointing. I ran tthis test on a Win32 machine and
without the Atlas library.

Why is there such a large difference in speed and how can I improve
the speed of randn in Python! Any help with this matter is truly
appreciated since I really would like to get away from Matlab and move
over to Python instead.

Yours Carl
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Eclipse Carriage Return Workaround

[Fabio Zadrozny]

On Fri, Dec 18, 2009 at 1:38 PM, Steve Holden  wrote:
> I've written a Python 3 course that uses an Eclipse-based teaching
> system. The school is telling me that their version of Eclipse/pydev
> appears to have an input() function that appends a carriage return
> character to the user's input. This makes several things go screwy, as
> it's definitely not the way the standalone interpreter works, even on
> Windows.
>
> Can anyone think of a simple way work around this issue by overriding
> __builtins__.input() with a function that calls input() and then returns
> an rstrip()ped version of the input string? I though of setting a
> PYTHONSTARTUP environment variable, but that only affects interactive
> interpreter instances.

In my opinion that's a python bug (because it should be able to remove
the \r\n and not only \n).

Anyway, Pydev also had that problem and it was fixed by having a
custom sitecustomize.py:

See: 
http://github.com/aptana/Pydev/tree/master/plugins/org.python.pydev/PySrc/pydev_sitecustomize/

It just has to added to the pythonpath before the run (and it'll
remove itself and call the default later on) -- the only catch is that
it has to be on a folder called "pydev_sitecustomize" -- you can
probably change the code if you don't want to follow that.

It'll fix input(), raw_input() and will also fix encoding problems
when writing non ASCII to the console (you may set a
'PYDEV_CONSOLE_ENCODING' in the environment or let it try to find a
default on) -- should be compatible with python 2 or 3.

Cheers,

Fabio
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyArg_ParseTupleAndKeywords in Python3.1

[Emeka]

Okay if that is the case, why do we need it? By having int a = 65, b = 66 ,
why should we also have *kwlist[]?



static PyObject* foo(PyObject *self, PyObject *args, PyObject *kwrds)
{
   int a=65, b=66;
   char *kwlist[] = {"a", "b", NULL};
   if (!PyArg_ParseTupleAndKeywords(args, kwrds, "|CC", kwlist, &a,
&b))
   return NULL;
   return Py_BuildValue("(CC)", a, b);
}

On Fri, Dec 18, 2009 at 9:31 PM, casevh  wrote:

> On Dec 18, 10:28 am, Joachim Dahl  wrote:
> > My mistake seems to be that I declared
> >
> > char a, b;
> >
> > instead of
> >
> > int a, b;
> >
> > Thank you for sorting this out.
> >
> > Joachim
>
> I think you need to initialize them, too.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Eclipse Carriage Return Workaround

[Fabio Zadrozny]

On Sat, Dec 19, 2009 at 8:36 AM, Fabio Zadrozny  wrote:
> On Fri, Dec 18, 2009 at 1:38 PM, Steve Holden  wrote:
>> I've written a Python 3 course that uses an Eclipse-based teaching
>> system. The school is telling me that their version of Eclipse/pydev
>> appears to have an input() function that appends a carriage return
>> character to the user's input. This makes several things go screwy, as
>> it's definitely not the way the standalone interpreter works, even on
>> Windows.
>>
>> Can anyone think of a simple way work around this issue by overriding
>> __builtins__.input() with a function that calls input() and then returns
>> an rstrip()ped version of the input string? I though of setting a
>> PYTHONSTARTUP environment variable, but that only affects interactive
>> interpreter instances.
>
> In my opinion that's a python bug (because it should be able to remove
> the \r\n and not only \n).
>
> Anyway, Pydev also had that problem and it was fixed by having a
> custom sitecustomize.py:
>
> See: 
> http://github.com/aptana/Pydev/tree/master/plugins/org.python.pydev/PySrc/pydev_sitecustomize/
>
> It just has to added to the pythonpath before the run (and it'll
> remove itself and call the default later on) -- the only catch is that
> it has to be on a folder called "pydev_sitecustomize" -- you can
> probably change the code if you don't want to follow that.
>
> It'll fix input(), raw_input() and will also fix encoding problems
> when writing non ASCII to the console (you may set a
> 'PYDEV_CONSOLE_ENCODING' in the environment or let it try to find a
> default on) -- should be compatible with python 2 or 3.
>

I just noted that you said they are already using pydev -- maybe it's
an old version? Or maybe you're doing a custom launcher that overrides
the usual pythonpath and for some reason is not passing the
pydev_sitecustomize folder?

Cheers,

Fabio
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Seek support for new slice syntax PEP.

[Colin W.]

On 18-Dec-09 23:16 PM, Nobody wrote:

On Fri, 18 Dec 2009 09:49:26 -0500, Colin W. wrote:


You don't say, but seem to imply that the slice components include None.


That's how missing components are implemented at the language level:

>  class foo:
=   def __getitem__(self, s):
= return s
=
>  x = foo()
>  x[::]
slice(None, None, None)
>  x[1::2]
slice(1, None, 2)

The defaults of zero, sys.maxint and one apply to built-in types, but
nothing forces user-defined types to behave this way.

Or maybe I misunderstood your point.


No, it seems that the implementation is a little different from the doc.

You are right:
*** Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit 
(Intel)] on win32. ***

>>> a= range(10)
>>> a[2:8:2]
[2, 4, 6]
>>> a[2::2]
[2, 4, 6, 8]
>>> a[2:None:2]
[2, 4, 6, 8]
>>>
I had expected the last to be rejected, but it fits with the overall 
philosophy.


Colin W
--
http://mail.python.org/mailman/listinfo/python-list

py itertools?

[mattia]

Hi all, I need to create the permutation of two strings but without 
repeat the values, e.g. 'ab' for me is equal to 'ba'. Here is my 
solution, but maybe the python library provides something better:

>>> def mcd(a, b):
... if b == 0:
... return a
... else:
... return mcd(b, a % b)
...
>>> def mcm(a, b):
... return int((a * b) / mcd(a, b))
...
>>> s1 = 'abc'
>>> s2 = 'wt'
>>> m = mcm(len(s1), len(s2))
>>> set(zip(s1*m, s2*m))
{('a', 'w'), ('a', 't'), ('b', 'w'), ('c', 't'), ('b', 't'), ('c', 'w')}

Any help?

Thanks, Mattia
-- 
http://mail.python.org/mailman/listinfo/python-list

comparing dialects of csv-module

[Malte Dik]

Hi out there!

I want to put some intelligence into a csv reading script and in order to do 
so I want to compare possible different dialects I collect with some random

d = csv.Sniffer().sniff("1,2,3,4"),

because the csv is kinda dirty.

Now sniff() returns a class object and those aren't comparable in the "if 
dialect_1 == dialect_2: count something" sense.

Has anyone around here already dealt with this kind of problem and maybe 
even a solution I could utilize? That would be great.

If not - I guess I would just write a quick function comparing the 
attributes of those dialects ... - but just out of educational curiosity: 
Would it be the right way to implement an __eq__(...) function into the 
Dialect class or how would someone who would want to make it right do it?


Sincerest greetings,

Malte
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Steven D'Aprano]

On Sat, 19 Dec 2009 02:05:17 -0800, Carl Johan Rehn wrote:

> Dear friends,
> 
> I plan to port a Monte Carlo engine from Matlab to Python. However, when
> I timed randn(N1, N2) in Python and compared it with Matlab's randn,

What's randn? I don't know that function. I know the randint, random, and 
randrange functions, but not randn. Where does it come from?


> Matlab came out as a clear winner with a speedup of 3-4 times. This was
> truly disappointing. I ran tthis test on a Win32 machine and without the
> Atlas library.
> 
> Why is there such a large difference in speed and how can I improve the
> speed of randn in Python! Any help with this matter is truly appreciated
> since I really would like to get away from Matlab and move over to
> Python instead.

Could be many reasons. Python could be generally slower than Matlab. Your 
timing code might have been faulty and you weren't comparing equal 
amounts of work (if your Python code was doing four times as much work as 
the Matlab code, then naturally it will be four times slower). Perhaps 
the Matlab random number generator is a low-quality generator which is 
fast but not very random. Python uses a very high quality RNG which is 
not cheap.

But does it really matter if the RNG is slower? Your Monte Carlo engine 
is a lot more than just a RNG. What matters is whether the engine as a 
whole is faster or slower, not whether one small component is slower.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating Classes

[Emeka]

Hello Dave

>
> There are more complex things that can go on, like creating "bound"
> function objects, but  I think this should get you pretty far.
>
> Could explain the complex things for me?

 Regards,
Janus
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using ZODB (or something else) for storing lots of metadata about RSS/Atom feeds and posts

[Bruno Desthuilliers]

Thomas Doggette a écrit :
> I'm working on (or rather, at this point, planning) a project that
> will involve keeping track of every post in a large number of Atom
> feeds, as well as a lot of metadata about how posts are linked and how
> users interact with them.
> 
> The idea of having all of these be persistent objects is very
> appealing, but I'm not sure the ZODB is ideal for what I'm doing. Can
> people with more experience working with data sets like this give me
> some advice on what they've had luck using?

My own experience with the ZODB for large number of small objects, lot
of metadata and indexes, frequent updates, and querying the whole thing,
has been at least very disappointing.

RDBMS are possibly not the perfect solution for each and any kind of
problem datastructures, but unless you have to deal with complex
heterogenous trees or graphs, they sure are a safe bet for most data
storage, retrieval and analysis jobs.

Also, I don't have ant experience with Google's AppEngine's datastore
nor Apache CouchDB, but given the description of your project they might
be appropriate solutions.

My 2 cents.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help with invoking standard mail app in windows

[Kev Dwyer]

On Sat, 19 Dec 2009 06:36:32 +1100, Astan Chee wrote:

> Kev Dwyer wrote:
>> Hello Astan,
>>
>> Your code executes without error for me on Win98 (!) with Python 2.5 or
>> XP with Python 2.6.
>>
>> It would help people to help you if you could provide the *exact*
>> console output from when you try to execute the code, *including* the
>> traceback. That way we can work out which line of code is hitting the
>> exception.
>>
>> Cheers,
>>
>> Kev
>>
>>
> Hi,
> My mistake. The length of body is over 1400 characters. Here is my
> updated code and result:
> 
> import urllib, webbrowser, win32api
> def mailto_url(to=None,subject=None,body=None,cc=None):
> """
> encodes the content as a mailto link as described on
> http://www.faqs.org/rfcs/rfc2368.html """ url = "mailto: " +
> urllib.quote(to.strip(),"@,") sep = "?"
> if cc:
> url+= sep + "cc=" + urllib.quote(cc,"@,") sep = "&"
> if subject:
> url+= sep + "subject=" + urllib.quote(subject,"") sep = "&"
> if body:
> # Also note that line breaks in the body of a message MUST be #
> encoded with "%0D%0A". (RFC 2368)
> body="\r\n".join(body.splitlines())
> url+= sep + "body=" + urllib.quote(body,"") sep = "&"
> return url
> 
> txtTo = "t...@com.com"
> txtSubject = "Test Subject"
> body = "Test body"
> for t in range(278):
> body+="test "
> txtCC = "cc_t...@com.com"
> 
> url = mailto_url(txtTo,txtSubject,body,txtCC)
> #win32api.ShellExecute(0,'open',url,None,None,0)
> webbrowser.open(url,new=1)
> # os.startfile(url)
> 
> result:
> 
> Traceback (most recent call last):
>   File "C:/stanc_home/python/mail_test.py", line 32, in 
> webbrowser.open(url,new=1)
>   File "C:\Python25\lib\webbrowser.py", line 61, in open
> if browser.open(url, new, autoraise):
>   File "C:\Python25\lib\webbrowser.py", line 518, in open
> os.startfile(url)
> WindowsError: [Error 5] Access is denied: 'mailto:
> t...@com.com?cc=cc_test@com.com&subject=Test%20Subject&body=Test%

> 
> Is there some sort of limitation here? If I shorten the string, it works
> fine. You're right, but I'm wondering if there is a way to go around
> this limitation.
> Thanks again
> Cheers
> Astan


Hello Astan,

After a bit of experimentation I find I get the same problem on XP using 
2.6 for len(body) > 1973, using the os.startfile method.  Some light 
googling suggests this is probably a limit within Outlook itself.

If you just want a way to start Outlook programmatically you may be able 
to work around this by automating Outlook - try googling python outlook 
automation.

If you want to start a default mail client on any machine using the 
mailto protocol then you'll have to limit the size of your message 
bodies.  You may find that other mail clients have limits too - I doubt 
that any client (or browser) will accept urls of unlimited size.

Cheers,

Kev

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: py itertools?

[Chris Rebert]

On Sat, Dec 19, 2009 at 2:54 AM, mattia  wrote:
> Hi all, I need to create the permutation of two strings but without
> repeat the values, e.g. 'ab' for me is equal to 'ba'. Here is my
> solution, but maybe the python library provides something better:
>
 def mcd(a, b):
> ...     if b == 0:
> ...         return a
> ...     else:
> ...         return mcd(b, a % b)
> ...
 def mcm(a, b):
> ...     return int((a * b) / mcd(a, b))
> ...
 s1 = 'abc'
 s2 = 'wt'
 m = mcm(len(s1), len(s2))
 set(zip(s1*m, s2*m))
> {('a', 'w'), ('a', 't'), ('b', 'w'), ('c', 't'), ('b', 't'), ('c', 'w')}
>
> Any help?

Surprised you didn't think of the seemingly obvious approach:

def permute_chars(one, two):
for left in set(one):
for right in set(two):
yield (left, right)

>>> list(permute_chars('abc', 'wt'))
[('a', 'w'), ('a', 't'), ('b', 'w'), ('b', 't'), ('c', 'w'), ('c', 't')]

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Carl Johan Rehn]

On Dec 19, 12:29 pm, Steven D'Aprano  wrote:
> On Sat, 19 Dec 2009 02:05:17 -0800, Carl Johan Rehn wrote:
> > Dear friends,
>
> > I plan to port a Monte Carlo engine from Matlab to Python. However, when
> > I timed randn(N1, N2) in Python and compared it with Matlab's randn,
>
> What's randn? I don't know that function. I know the randint, random, and
> randrange functions, but not randn. Where does it come from?
>
> > Matlab came out as a clear winner with a speedup of 3-4 times. This was
> > truly disappointing. I ran tthis test on a Win32 machine and without the
> > Atlas library.
>
> > Why is there such a large difference in speed and how can I improve the
> > speed of randn in Python! Any help with this matter is truly appreciated
> > since I really would like to get away from Matlab and move over to
> > Python instead.
>
> Could be many reasons. Python could be generally slower than Matlab. Your
> timing code might have been faulty and you weren't comparing equal
> amounts of work (if your Python code was doing four times as much work as
> the Matlab code, then naturally it will be four times slower). Perhaps
> the Matlab random number generator is a low-quality generator which is
> fast but not very random. Python uses a very high quality RNG which is
> not cheap.
>
> But does it really matter if the RNG is slower? Your Monte Carlo engine
> is a lot more than just a RNG. What matters is whether the engine as a
> whole is faster or slower, not whether one small component is slower.
>
> --
> Steven

randn is given by

>> import numpy
>>> numpy.random.randn(2,3)
array([[-2.66435181, -0.32486419,  0.12742156],
   [-0.2387061 , -0.55894044,  1.20750493]])

Generally, at least in my MC application, I need a large number of
random numbers. Usually I execute, for example, r = randn(100, 1)
sequentially a relatively large number of times until sufficient
accuracy has been reached. Thus, randn is in my case a mission
critical component for obtaining an acceptable overall run time.
Matlab and numpy have (by chance?) the exact names for the same
functionality, so I was very happy with numpy's implementation until I
timed it. So the basioc question is, how can I speed up random number
generation?

Carl



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[sturlamolden]

On 19 Des, 11:05, Carl Johan Rehn  wrote:

> I plan to port a Monte Carlo engine from Matlab to Python. However,
> when I timed randn(N1, N2) in Python and compared it with Matlab's
> randn, Matlab came out as a clear winner with a speedup of 3-4 times.
> This was truly disappointing. I ran tthis test on a Win32 machine and
> without the Atlas library.

This is due to the algorithm. Matlab is using Marsaglia's ziggurat
method. Is is the fastest there is for normal and gamma random
variates. NumPy uses the Mersenne Twister to produce uniform random
deviates, and then applies trancendental functions to transform to the
normal distribution. Marsaglia's C code for ziggurat is freely
available, so you can compile it yourself and call from ctypes, Cython
or f2py.

The PRNG does not use BLAS/ATLAS.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[sturlamolden]

On 19 Des, 12:29, Steven D'Aprano  wrote:

> Perhaps
> the Matlab random number generator is a low-quality generator which is
> fast but not very random. Python uses a very high quality RNG which is
> not cheap.

Marsaglia and Matlab's implementation of ziggurat uses a slightly
lower quality RNG for uniform deviates than NumPy's Mersenne Twister.
But the real speed advantage comes from avoiding trancendental
functions. I have for some time thought of contributing a ziggurat
generator to NumPy, while retaining the Mersenne Twister internally,
but I have not got around to do it.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: comparing dialects of csv-module

[Peter Otten]

Malte Dik wrote:

> Hi out there!
> 
> I want to put some intelligence into a csv reading script and in order to
> do so I want to compare possible different dialects I collect with some
> random
> 
> d = csv.Sniffer().sniff("1,2,3,4"),
> 
> because the csv is kinda dirty.
> 
> Now sniff() returns a class object and those aren't comparable in the "if
> dialect_1 == dialect_2: count something" sense.
> 
> Has anyone around here already dealt with this kind of problem and maybe
> even a solution I could utilize? That would be great.

An implementation for the lazy

>>> import csv
>>> d = csv.Sniffer().sniff("1,2,3")
>>> def eq(a, b, attributes=[name for name in dir(d) if not 
name.startswith("_")]):
... return all(getattr(a, n, None) == getattr(b, n, None) for n in 
attributes)
...
>>> eq(d, csv.Sniffer().sniff("3,4,5"))
True
>>> eq(d, csv.Sniffer().sniff("'3','4','5'"))
False
>>> eq(d, csv.Sniffer().sniff("3;4;5"))
False
>>> eq(d, csv.Sniffer().sniff("3,4,' 5'"))
True

 
> If not - I guess I would just write a quick function comparing the
> attributes of those dialects ... - but just out of educational curiosity:
> Would it be the right way to implement an __eq__(...) function into the
> Dialect class or how would someone who would want to make it right do it?

I don't know if you can do it for classic classes; for newstyle classes 
you'd have to reimplement comparison in the metaclass:

>>> class Dialect:
... class __metaclass__(type):
... def __eq__(self, other):
... return self._key() == other._key()
... def __ne__(self, other):
... return self._key() != other._key()
... def _key(self):
... return self.quotechar, self.delimiter #,...
...
>>> class A(Dialect):
... quotechar = "'"
... delimiter = ":"
...
>>> class B(Dialect):
... quotechar = "'"
... delimiter = ","
...
>>> A == B
False
>>> B.delimiter = ":"
>>> A == B
True

On a side note, I think it's a C++ism that dialects are classes rather than 
class instances. That's a superfluous complication since in Python no work 
will be moved from compile time to runtime anyway.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[sturlamolden]

On 19 Des, 14:06, Carl Johan Rehn  wrote:

> Matlab and numpy have (by chance?) the exact names for the same
> functionality,

Common ancenstry, NumPy and Matlab borrowed the name from IDL.


LabView, Octave and SciLab uses the name randn as well.


> So the basioc question is, how can I speed up random number
> generation?

The obvious thing would be to compile ziggurat yourself, and turn on
optimization flags for your hardware.
http://www.jstatsoft.org/v05/i08/


P.S. Be careful if you consider using more than one processor.
Multithreading is a very difficult issue with PRNGs, becuase it is
difficult to guarrantee they are truely independent. But you can use a
producer-consumer pattern, though: one thread constantly producing
random numbers (writing into a buffer or pipe) and another thread(s)
consuming them.












-- 
http://mail.python.org/mailman/listinfo/python-list

Re: shouldn't list comprehension be faster than for loops?

[sturlamolden]

On 19 Des, 02:28, Ryan Kelly  wrote:

> Not so.  If you use the "dis" module to peek at the bytecode generated
> for a list comprehension, you'll see it's very similar to that generated
> for an explicit for-loop.  The byte-code for a call to map is very
> different.

First, you failed to realize that the bytecode is different because
map is doing the work in C.

Second, you did not provide bytecode for the for-loop.









-- 
http://mail.python.org/mailman/listinfo/python-list

Re: py itertools?

[Lie Ryan]

On 12/19/2009 11:48 PM, Chris Rebert wrote:


Surprised you didn't think of the seemingly obvious approach:

def permute_chars(one, two):
 for left in set(one):
 for right in set(two):
 yield (left, right)


list(permute_chars('abc', 'wt'))

[('a', 'w'), ('a', 't'), ('b', 'w'), ('b', 't'), ('c', 'w'), ('c', 't')]



even less work:
import itertools
print set(itertools.product('abc', 'wt'))

but neither of those two solves the OP's problem. And neither the OP's 
own solution solves his own problem (per my understanding from his 
description).


what he wanted was something like:
print set(tuple(sorted(x)) for x in itertools.product(s1, s2))


or, just for some functional fun, when written in point-free form:

from itertools import product
from functools import partial
def compose(f, g):
return lambda *a, **k: f(g(*a, **k))

sortedtuple = compose(tuple, sorted)
setcomp = compose(set, map)
unique_tuples = partial(setcomp, sortedtuple)
permute_chars = compose(unique_tuples, product)
print permute_chars(s1, s2)
--
http://mail.python.org/mailman/listinfo/python-list

Re: comparing dialects of csv-module

[Malte Dik]

quote:
> An implementation for the lazy
> 
 import csv
 d = csv.Sniffer().sniff("1,2,3")
 def eq(a, b, attributes=[name for name in dir(d) if not
> name.startswith("_")]):
> ... return all(getattr(a, n, None) == getattr(b, n, None) for n in
> attributes)
> ...

Wow, this is awesome. I'd never come up with something like this.


Will digg into it deeper as I implement it (code snippets like this need to 
melt in order to effuse all their flavor ;), but want to thank you very much 
in the first place! :)



Have a nice day everyone,

Malte
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Carl Johan Rehn]

On Dec 19, 2:49 pm, sturlamolden  wrote:
> On 19 Des, 11:05, Carl Johan Rehn  wrote:
>
> > I plan to port a Monte Carlo engine from Matlab to Python. However,
> > when I timed randn(N1, N2) in Python and compared it with Matlab's
> > randn, Matlab came out as a clear winner with a speedup of 3-4 times.
> > This was truly disappointing. I ran tthis test on a Win32 machine and
> > without the Atlas library.
>
> This is due to the algorithm. Matlab is using Marsaglia's ziggurat
> method. Is is the fastest there is for normal and gamma random
> variates. NumPy uses the Mersenne Twister to produce uniform random
> deviates, and then applies trancendental functions to transform to the
> normal distribution. Marsaglia's C code for ziggurat is freely
> available, so you can compile it yourself and call from ctypes, Cython
> or f2py.
>
> The PRNG does not use BLAS/ATLAS.

Thank you, this was very informative. I know about the Mersenne
Twister, but had no idea about Marsaglia's ziggurat method. I will
definitely try f2py or cython.

Well, I guess I knew that random numbers were not handled by BLAS/
ATLAS, but wasn't 100% sure.

Carl
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How Do I...?

[Tim Chase]

Victor Subervi wrote:

On Fri, Dec 18, 2009 at 3:03 PM, Tim Chase wrote:

Well, you start by reading a book on how to program.  You would then learn
that what you want (in all likelihood) is a dictionary/map structure for
dynamically created key/value pairs. Once you have progressed from your
current apprenticeship and achieved the rank of third-degree journeyman
programmer, the ways of dynamic variable creation will avail themselves.


Why the arrogance? Why talk down to me?


The aim was not arrogance, but expression of exasperation at your 
"using c.l.p to write my code and interpret my error messages" 
programming style that seems to indicate that you don't 
understand how to use tools like your editor, traceback messages, 
& google; or that you've not invested the time to learn 
programming concepts such as when to use dictionaries/maps, HTML, 
HTTP/server stuff, SMTP/email stuff, or research hosting provider 
information.


Let's take a tour back through some of your previous posts to the 
list to see why the exasperation:


Just in my readily-available offline archives, I count at least 4 
off-topic (non-Python) posts, three of which you KNEW were OT 
because you put "OT" in the subject[1].  The 4th[2] is just a 
google-search away.  Any basic HTML tutorial on using form 
elements would show you how to do this.  And on the topic of 
googling for answers you had questions about good mailers[3], a 
quick search for "python send mail" turns up a multitude of 
built-in and code-copyable solutions.


You don't seem to learn from previous answers[4] where the 
solution in the first response was "make sure you're not masking 
the built-in module with a same-named file in your project 
directory", only to have the same issue in the 2nd email.  While 
it's somewhat forgivable as I too have accidentally masked my 
ability to send email by having an "email" module in the local 
directory, I learned to google the traceback's core message 
"ImportError: No module named base64MIME" which gave me the answer.


You don't seem to read tracebacks[5].  When you do get tracebacks 
that you don't understand, many of your initial posts either post 
them in some funky line-numbered format[6] that makes them hard 
to read (though can be helpful, so this isn't as grievous)), or 
you simply omit the traceback completely[7].  Even if asked 
explicitly for them[8].  They contain valuable information.  Your 
omission of them frustrates anybody trying to help.  There's the 
omission of key information[9] or improperly hand-transcribing 
code instead of copy-and-pasting the actual code.


And that doesn't even touch on the issues of repeated top-posting 
which is just annoying.  Folks have been pretty lax, giving you 
subtle "changed to inline format to make it easier to follow", 
but you don't seem to pick up on the suggestion.  I'll make it 
explicit:  post replies inline for best results.


You'll find that comp.lang.python is a generally friendly place, 
but HELP US HELP YOU.  Try doing your own research first, reading 
error messages, giving us the tools we need to help you, and 
adhering to conventions like inline posting.


I'm glad if my underlying suggestion of using a dict helped you 
reach a solution, but would be far more glad if you'd take this 
message to heart -- not as an enumeration transgressions, but as 
a way you can better ingratiate yourself with the list.


-tkc


[1]
OT Virtual Server Host
(OT) Recommend FTP Client
(OT) Where Are Cookies Stored

[2] Workaround to Add values to TextArea

[3] A Good Mailer

[4]
Can't Find Module
Python Will Not Send Email!!

[5] Switching Databases

[6] Problem w/ smtplib

[7]
Calendar Problem
Calendar Stuff
...and others

[8] Switching Databases

[9]
Nested Dicts
Calendar Stuff






--
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Carl Johan Rehn]

On Dec 19, 3:16 pm, sturlamolden  wrote:
> On 19 Des, 14:06, Carl Johan Rehn  wrote:
>
> > Matlab and numpy have (by chance?) the exact names for the same
> > functionality,
>
> Common ancenstry, NumPy and Matlab borrowed the name from IDL.
>
> LabView, Octave and SciLab uses the name randn as well.
>
> > So the basioc question is, how can I speed up random number
> > generation?
>
> The obvious thing would be to compile ziggurat yourself, and turn on
> optimization flags for your hardware.http://www.jstatsoft.org/v05/i08/
>
> P.S. Be careful if you consider using more than one processor.
> Multithreading is a very difficult issue with PRNGs, becuase it is
> difficult to guarrantee they are truely independent. But you can use a
> producer-consumer pattern, though: one thread constantly producing
> random numbers (writing into a buffer or pipe) and another thread(s)
> consuming them.

How about mulit-core or (perhaps more exciting) GPU and CUDA? I must
admit that I am extremely interested in trying the CUDA-alternative.

Obviously, cuBLAS is not an option here, so what is the safest route
for a novice parallel-programmer?

Carl


Carl
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Steven D'Aprano]

On Sat, 19 Dec 2009 05:06:53 -0800, Carl Johan Rehn wrote:

> so I was very happy with numpy's implementation until I timed it.

How did you time it?


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[sturlamolden]

On 19 Des, 16:20, Carl Johan Rehn  wrote:

> How about mulit-core or (perhaps more exciting) GPU and CUDA? I must
> admit that I am extremely interested in trying the CUDA-alternative.
>
> Obviously, cuBLAS is not an option here, so what is the safest route
> for a novice parallel-programmer?

The problem with PRNG is that they are iterative in nature, and
maintain global states. They are therefore very hard to vectorize. A
GPU will not help. The GPU has hundreds of computational cores that
can run kernels, but you only get to utilize one.

Parallel PRNGs are an unsolved problem in computer science.





-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Seek support for new slice syntax PEP.

[Dave Angel]

Colin W. wrote:
On 
18-Dec-09 23:16 PM, Nobody wrote:

On Fri, 18 Dec 2009 09:49:26 -0500, Colin W. wrote:

You don't say, but seem to imply that the slice components include 
None.


That's how missing components are implemented at the language level:

> class foo:
= def __getitem__(self, s):
= return s
=
> x = foo()
> x[::]
slice(None, None, None)
> x[1::2]
slice(1, None, 2)

The defaults of zero, sys.maxint and one apply to built-in types, but
nothing forces user-defined types to behave this way.

Or maybe I misunderstood your point.


No, it seems that the implementation is a little different from the doc.

You are right:
*** Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 
bit (Intel)] on win32. ***

>>> a= range(10)
>>> a[2:8:2]
[2, 4, 6]
>>> a[2::2]
[2, 4, 6, 8]
>>> a[2:None:2]
[2, 4, 6, 8]
>>>
I had expected the last to be rejected, but it fits with the overall 
philosophy.


Colin W



None is perfectly valid as a parameter to a slice. To quote the 2.6.4 
docs, in section 6.6:


The slice of /s/ from /i/ to /j/ with step /k/ is defined as the 
sequence of items with index x = i + n*k such that 0 <= n < (j-i)/k. In 
other words, the indices are i, i+k, i+2*k, i+3*k and so on, stopping 
when /j/ is reached (but never including /j/). If /i/ or /j/ is greater 
than len(s), use len(s). If /i/ or /j/ are omitted or None, they become 
“end” values (which end depends on the sign of /k/). Note, /k/ cannot be 
zero. If /k/ is None, it is treated like 1.



DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: py itertools?

[mattia]

Il Sat, 19 Dec 2009 10:54:58 +, mattia ha scritto:

> Hi all, I need to create the permutation of two strings but without
> repeat the values, e.g. 'ab' for me is equal to 'ba'. Here is my
> solution, but maybe the python library provides something better:
> 
 def mcd(a, b):
> ... if b == 0:
> ... return a
> ... else:
> ... return mcd(b, a % b)
> ...
 def mcm(a, b):
> ... return int((a * b) / mcd(a, b)) ...
 s1 = 'abc'
 s2 = 'wt'
 m = mcm(len(s1), len(s2))
 set(zip(s1*m, s2*m))
> {('a', 'w'), ('a', 't'), ('b', 'w'), ('c', 't'), ('b', 't'), ('c', 'w')}
> 
> Any help?
> 
> Thanks, Mattia

Well, this is the code I'm using:

import itertools

def cheapest_travel(l):
"""Given a list of departure and return dates, return the cheapest 
solution"""
s = set(itertools.permute(l[0], l[1]))
return sorted(t, key = lambda s: s[0][2] + s[1][2])

# example using a dict
d = {
'2009/12/21' : [[('d', 1, 2), ('d', 3, 4), ('d', 2, 3)], [('r', 3, 
5), ('r', 3, 8)]],
'2009/12/19' : [[('d', 1, 2), ('d', 2, 3)], [('r', 1, 4), ('r', 6, 
4), ('r', 3, 5), ('r', 3, 8)]],
'2009/12/23' : [[('d', 2, 5), ('d', 2, 4)], [('r', 4, 5)]],
'2009/12/26' : [[('d', 2, 5), ('d', 1, 4)], [('r', 3, 6)]],
'2009/12/28' : [[('d', 2, 5)], [('r', 4, 4)]]
}   

for k, v in d.items():
print(k)
res = cheapest_travel(v)
for x in res:
print(x[0], "-->", x[1], "cost", x[0][2] + x[1][2], "EUR")
-- 
http://mail.python.org/mailman/listinfo/python-list

how do I set a Python installation as the default under windows ?

[Stef Mientki]

hello,

I just upgraded from Python 2.5 to 2.6.
Most of the things work,
but I'm struggling with one issue,
when I start Python in a command window,
it still uses Python 2.5.

Is there a way to get Python 2.6 as my default Python environment ?

thanks,
Stef Mientki
--
http://mail.python.org/mailman/listinfo/python-list

Re: How Do I...?

[Victor Subervi]

On Sat, Dec 19, 2009 at 10:22 AM, Tim Chase
wrote:

> Victor Subervi wrote:
>
>> On Fri, Dec 18, 2009 at 3:03 PM, Tim Chase > >wrote:
>>
>>  Well, you start by reading a book on how to program.  You would then
>>> learn
>>> that what you want (in all likelihood) is a dictionary/map structure for
>>> dynamically created key/value pairs. Once you have progressed from your
>>> current apprenticeship and achieved the rank of third-degree journeyman
>>> programmer, the ways of dynamic variable creation will avail themselves.
>>>
>>
>> Why the arrogance? Why talk down to me?
>>
>
> The aim was not arrogance, but expression of exasperation


Exasperation is what triggers your arrogance. Learn to control it. "Walk a
mile in my mocassins." You can't do it. I'm an artist. I think out of my
right hemisphere, not my left like you. You couldn't possibly understand.
Yes, sometimes I chide myself for the questions I ask when I see the answers
come back. The problem is that I quite literally can't think like you. I
have to force myself to do it every time. To you it's as natural as
breathing, which is why you can't relate.

Thank you for your help anyway. Thank you for your patience. Please try to
understand. It starts by understanding you can't understand. You have my
continued promise that I will do all I can to edit my questions as
intelligently as you would before I post them. Trust me, I don't like
looking foolish, and I know I do. You should recognize that that alone is
chiding enough. It doesn't stop me, however, for continuing to program. The
whole universe is a program. I program to understand it in a way you
couldn't even begin to grasp.

BTW, although I know full well about dictionaries, I didn't know one could
use them like you indicated, nor would I have known how to google it. I did
try googling, FYI.
V
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Carl Johan Rehn]

On Dec 19, 4:47 pm, sturlamolden  wrote:
> On 19 Des, 16:20, Carl Johan Rehn  wrote:
>
> > How about mulit-core or (perhaps more exciting) GPU and CUDA? I must
> > admit that I am extremely interested in trying the CUDA-alternative.
>
> > Obviously, cuBLAS is not an option here, so what is the safest route
> > for a novice parallel-programmer?
>
> The problem with PRNG is that they are iterative in nature, and
> maintain global states. They are therefore very hard to vectorize. A
> GPU will not help. The GPU has hundreds of computational cores that
> can run kernels, but you only get to utilize one.
>
> Parallel PRNGs are an unsolved problem in computer science.


>>>How did you time it?

Well, in Matlab I used "tic; for i = 1:1000, randn(100, 1), end;
toc" and in IPython i used a similar construct but with "time" instead
of tic/(toc.


>>> Parallel PRNGs are an unsolved problem in computer science.

Thanks again for sharing your knowledge. I had no idea. This means
that if I want to speed up my application I have to go for the fastest
random generator and focus on other parts of my code that can be
vectorized.

Carl
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: iterators and views of lists

[Anh Hai Trinh]

On Dec 19, 5:47 am, Bearophile  wrote:

> It seems you have missed my post, so here it is, more explicitly:
>
> http://www.boostcon.com/site-media/var/sphene/sphwiki/attachment/2009...


Interestingly, my `listagent` can be used as a lazy iterator and thus
using itertools we can compose them just like Andrei's `range`.

the stage:

  x = range(0, 20, 2); x
  [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

  y = range(10, 20); y
  [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]

  z = [996, 758, 670, 236, 898, 337, 442, 452, 490, 547]

  from listagent import listagent
  import itertools

chain:

  sorted(itertools.chain(listagent(x)[::2], listagent(y)[-1:1:-2]))
  [0, 4, 8, 12, 13, 15, 16, 17, 19]

zip:

  sorted(itertools.izip(listagent(z)[1::3], listagent(x)[2::3]))
  [(452, 16), (758, 4), (898, 10)]

stride: slicing an agent returns another one, those lazy agents!

  python -m timeit -s'from listagent import listagent' 'list(listagent
(range(100))[1:-1:5][::7][1:-1:42])'
  10 loops, best of 3: 55.5 msec per loop

  python -m timeit 'range(100)[1:-1:5][::7][1:-1:42]'
  10 loops, best of 3: 280 msec per loop

radial: not implemented


`listagent` is implemented in pure Python, of course. If implemented
in C, there is surprisingly little difference between a slicing
`agent` like this, or a listiterator, since both will have to keep a
pointer to an memory index of in the original list: the listiterator
advances this pointer when we call next(), whereas the agent keeps a
pointer to the starting element of the slice and return basically
(PyObject *) *(p+(i*step)) on __getitem__[i]. Mutability is a just
matter of exposing this pointer to Python code in a nice way.

aht
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating Classes

[Steve Holden]

Dave Angel wrote:
> seafoid wrote:
>> Hey Guys,
>>
>> I have started to read over classes as a brief respite from my parsing
>> problem.
>>
>> When a class is defined, how does the class access the data upon which
>> the
>> class should act?
>>
>> Example:
>>
>> class Seq:   
>> def __init__(self, data, alphabet = Alphabet.generic_alphabet):
>> self.data = data self.alphabet = alphabet
>>
>> def
>> tostring(self):  
>> return self.data 
>> def tomutable(self):
>> return MutableSeq(self.data, self.alphabet)
>> def count(self, item):
>> return len([x for x in self.data if x == item])
>>
>> I know what it should do, but have no idea how to feed it the data.
>>
>> Methinks I need to invest in actual computing books as learning from
>> biologists is hazy!
>>
>> Kind regards,
>> Seafoid.
>> 
> Steve's message was good, but I feel he kind of jumped in the middle.  A
> class is a description of a type of object, and the behaviors and data
> that each instance of the object supports.
> 
> You create the object by using the class name like a function call.  The
> arguments to that "call" are passed to the __init__ method.
> 
> So  obj = Seq("abcd")or   obj = Seq("defg", "abcdefg")would each
> create an object of the class.  But generally, many objects will exist,
> each with different data.
> 
> The data in the object is accessed in what appears to be the "self"
> namespace.  The name self is just a convention, but it's the first
> argument of each method of the class.  So when somebody calls the
> count() method, they pass 'item' a value, but self is used to refer to
> that particular object's data.
> 
> So how does 'self' get assigned?  That's what Steve was describing. 
> When you use the syntax:
> obj.count("value")
> 
> you actually call the count method with self referring to "obj" and item
> referring to "value".  obj does double-duty here, both defining which
> class' count() method will be called, and also supplying the first
> parameter to the call, the "self" parameter.
> 
> There are more complex things that can go on, like creating "bound"
> function objects, but  I think this should get you pretty far.
> 
> One other point:  you should always derive a class from some other
> class, or 'object' by default.  So you should being the class definition
> by:
> 
> class Seq(object):
> 
> Why?  It mainly has to do with super().  But in any case if you omit the
> 'object' it's an "old style" class, and that's not even supported in
> 3.x, so it's better to just get in the habit before it matters.
> 
With respect, unless you have to expound on the differences between the
old-style and the new-style classes (which aren't relevant here) you are
just introducing a red herring by even mentioning it. The average Python
user won't need to use super() in their first year as a Python programmer.

And, since you brought up Python 3, it's not necessary to explicitly
inherit from object to get new-style classes because, as you correctly
point out, old-style classes don't exist in Python 3.

I have no idea why you think "you should always derive a class from some
other class". That's pretty unnecessary.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programming intro book ch1 and ch2 (Windows/Python 3) - Request For Comments

[John Posner]

On Fri, 18 Dec 2009 13:00:48 -0500, Alf P. Steinbach   
wrote:




Chapter 2 is about Basic Concepts (of programming). It's the usual:  
variables, ...


1. Overall suggestion

You have a tendency to include non-pertinent asides [1]. But then,  
rambling a bit endows a manuscript with the author's distinctive voice.  
Fortunately, we live in a hypertext-enabled world, where you can have your  
cake and eat it, too. I suggest that you go over your manuscript with a  
ruthless eye, and turn your rambles into hypertext-accessible "sidebars".  
See how much you can reduce the length of Chapter 2, which current runs 98  
pages!


2. Comments on Section 2.6.7, "References & automatic garbage collection"

There's a spell-check evader on page 2:49: "trough" s.b. "through". And  
your spell-checker should have caught "adressable" on page 2:48.


I find your sequence-of-attribute-lookups approach to the topic of  
"variable assignment" interesting. The directed-graph illustration on page  
2:49 even hints at the fact that in the world of Python, names ("turtle",  
"forward", etc.) and objects (various kinds of yellow boxes) are very  
different things.


(I suggest getting rid of the callout "Very small fixed size variable".  
That way, only objects, not names, have the italicized callouts.)


But using the term "references" and the directed-graph metaphor has its  
drawbacks. Avoiding the term "reference" will make it easier to navigate  
the turbulent waters of call-by-reference vs. call-by-value vs.  
call-by-name. (You don't even stick a toe in those waters in Section  
2.7.5.) Combining memory addresses with the directed graph metaphor  
invites the reader to think at way too low a level, IMHO.


Another metaphor just occurred to me: a scavenger hunt. It even fits in  
with your potentially-infinite-attribute-access approach to the topic. A  
sequence of attribute accesses:


   turtle.forward.__doc__

... is like a sequence of scavenger-hunt instructions:

   1. Get your next clue at the big oak tree
   2. Get your next clue at the back door of the Valley Bank
   3. Get your next clue under Dad's Oldsmobile

It's clear that the scavenger hunt's clues (short characters strings --  
like Python names) are different from the physical objects that you access  
as the hunt progresses (tree, bank building, automobile -- like Python  
objects). I haven't lived with this metaphor long enough to form an  
opinion as to where it might reside on the brain-dead <---> brilliant  
scale.


As I've said in this forum (and the edu-sig forum) before, I think the  
best metaphor for understanding Python variable assignment is John Zelle's  
yellow-sticky-note metaphor. [2]


I hope these comments help.

-John

--
[1] Examples:

Section 2, page 2:1

It's almost impossible, but, as Robert A. Heinlein remarked,
"A Paradox May Be Paradoctored".

Section 2, page 2:3

(I believe the original text comes from the "Jargon file")
 about how arbitrary, undesirable and downright dangerous DWIM
 guessing can be: ...

Section 2.5.1, page 2:14

a natural extension is to make that into a square spiral with
far more windings; I recall it as a common theme in 1970’s
pop-art and it can be quite impressive!

Section 2.6.7, page 2:46

(some flat-Earthers once thought that the flat Earth rested
 on four turtles, which in turn stood on four larger
 turtles, and so on all the way down)

[2] "Python Programming: An Introduction to Computer Science" by John  
Zelle (Franklin, Biddle & Associates, 2004) See Section 2.5.1, "Simple  
Assignment"

--
http://mail.python.org/mailman/listinfo/python-list

Re: How Do I...?

[Rami Chowdhury]

On Dec 19, 2009, at 08:50 , Victor Subervi wrote:
> On Sat, Dec 19, 2009 at 10:22 AM, Tim Chase  
> wrote:
> Victor Subervi wrote:
> On Fri, Dec 18, 2009 at 3:03 PM, Tim Chase 
> wrote:
> 
> Well, you start by reading a book on how to program.  You would then learn
> that what you want (in all likelihood) is a dictionary/map structure for
> dynamically created key/value pairs. Once you have progressed from your
> current apprenticeship and achieved the rank of third-degree journeyman
> programmer, the ways of dynamic variable creation will avail themselves.
> 
> Why the arrogance? Why talk down to me?
> 
> The aim was not arrogance, but expression of exasperation
> You can't do it. [snip] You couldn't possibly understand. 

And you're accusing someone else of arrogance here ;-)? I don't think anyone is 
deliberately talking down to you -- would you contest the assertion that you 
are an 'apprentice' in the art of programming, and are posting on c.l.py to 
learn from those with more experience?

> You have my continued promise that I will do all I can to edit my questions 
> as intelligently as you would before I post them. 
> Trust me, I don't like looking foolish, and I know I do.

Tim's given you a few suggestions, as have many others -- perhaps it would be 
worth making a note, somewhere, of checks to go through before you post, to 
ensure that you come across as well as you intend?

HTH,
Rami






-- 
http://mail.python.org/mailman/listinfo/python-list

Tutorial: Accessing Desktop CouchDB with Python

[Terry Reedy]

Above is a section heading in
Code tutorial: make your application sync with Ubuntu One
http://arstechnica.com/open-source/guides/2009/12/code-tutorial-make-your-application-sync-with-ubuntu-one.ars

I just read part of it and thought others might find it interesting.

tjr

--
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Lie Ryan]

On 12/20/2009 4:02 AM, Carl Johan Rehn wrote:

How did you time it?


Well, in Matlab I used "tic; for i = 1:1000, randn(100, 1), end;
toc" and in IPython i used a similar construct but with "time" instead
of tic/(toc.


Code?


Parallel PRNGs are an unsolved problem in computer science.


Thanks again for sharing your knowledge. I had no idea. This means
that if I want to speed up my application I have to go for the fastest
random generator and focus on other parts of my code that can be
vectorized.


If you don't care about "repeatability" (which is already extremely 
difficult in parallel processing even without random number generators), 
you can just start two PRNG at two distinct states (and probably from 
two different algorithms) and they will each spews out two independent 
streams of random numbers. What was "unsolved" was the "pseudo-" part of 
the random number generation, which guarantee perfect replayability in 
all conditions.

--
http://mail.python.org/mailman/listinfo/python-list

Re: How Do I...?

[Victor Subervi]

On Sat, Dec 19, 2009 at 3:06 PM, Rami Chowdhury wrote:

> Tim's given you a few suggestions, as have many others -- perhaps it would
> be worth making a note, somewhere, of checks to go through before you post,
> to ensure that you come across as well as you intend?
>

I appreciate Tim's advice. I came across as I intended.
V
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[Yarko]

On Dec 19, 12:42 am, AppRe Godeck  wrote:
> Just curious if anybody prefers web2py over django, and visa versa. I
> know it's been discussed on a flame war level a lot. I am looking for a
> more intellectual reasoning behind using one or the other.

Chevy or Ford?  (or whatever pair you prefer)
vi or emacs?
...

These hold one aspect.

Hammer or a saw?

Hold (perhaps) another...

us.pycon.org, for example, uses both (in reality a mix of the above
argument sets, but at least evidence of the latter: different tools
for different problems).

>From a rapid prototyping perspective, web2py is heavily data-table
efficient: that is, you can define a system, and all the app creation,
form generation and validation have defaults out of the box, and you
can have a "sense" of your data-centric structure in minutes.   The
same argument can go against ("how do I get it to do exactly what _I_
want it to, not what it wants to?") - that is, defaults hide things,
and  that has two edges...

>From a layout/user interaction rapid prototyping perspective, web2py
is just entering the waters...

There is a steady growth of users, and (as you would expect for a
young framework), a lot of changes going on (although backward
compatiblity is a constant mantra when considering changes, that too
is a double-edged thing).

I find web2py useful, fast, and at times / in areas not as evolved /
flexible as I'd like.  BUT I could learn it quickly, and get to work
quickly.

I have taken an intro Django course (at a PyCon), have built a few
things with it (not nearly as many as I have w/ web2py), and I _can_
do things in it - so I'll let someone else w/ django "miles" under
their belt speak their mind.

- Yarko
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[Yarko]

On Dec 19, 2:48 pm, Yarko  wrote:
> On Dec 19, 12:42 am, AppRe Godeck  wrote:
>
> > Just curious if anybody prefers web2py over django, and visa versa. I
> > know it's been discussed on a flame war level a lot. I am looking for a
> > more intellectual reasoning behind using one or the other.
>
> Chevy or Ford?  (or whatever pair you prefer)
> vi or emacs?
> ...
>
> These hold one aspect.
>
> Hammer or a saw?
>
> Hold (perhaps) another...
>
> us.pycon.org, for example, uses both (in reality a mix of the above
> argument sets, but at least evidence of the latter: different tools
> for different problems).
>
> From a rapid prototyping perspective, web2py is heavily data-table
> efficient: that is, you can define a system, and all the app creation,
> form generation and validation have defaults out of the box, and you
> can have a "sense" of your data-centric structure in minutes.   The
> same argument can go against ("how do I get it to do exactly what _I_
> want it to, not what it wants to?") - that is, defaults hide things,
> and  that has two edges...
>
> From a layout/user interaction rapid prototyping perspective, web2py
> is just entering the waters...
>
> There is a steady growth of users, and (as you would expect for a
> young framework), a lot of changes going on (although backward
> compatiblity is a constant mantra when considering changes, that too
> is a double-edged thing).
>
> I find web2py useful, fast, and at times / in areas not as evolved /
> flexible as I'd like.  BUT I could learn it quickly, and get to work
> quickly.

Oh and one more thing: I find it dependable (not that snapshots don't
have bugs, but that they are well defined, not "wild", and quickly
fixed - and if you work around them, you can also depend on the system
you've created).  FYI, it does the money/registration part of PyCon
(past 2 years).

>
> I have taken an intro Django course (at a PyCon), have built a few
> things with it (not nearly as many as I have w/ web2py), and I _can_
> do things in it - so I'll let someone else w/ django "miles" under
> their belt speak their mind.
>
> - Yarko

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[sturlamolden]

On 19 Des, 21:26, Lie Ryan  wrote:

> you can just start two PRNG at two distinct states

No. You have to know for certain that the outputs don't overlap.

If you pick two random states (using any PRNG), you need error-
checking that states are always unique, i.e. that each PRNG never
reaches the starting state of the other(s). If I had to do this, I
would e.g. hash the state to a 32 bit integer. You can deal with
errors by jumping to a different state or simply aborting the
simulation. The problem is that the book-keeping will be costly
compared to the work involved in generating a single pseudorandom
deviate. So while we can construct a parallel PRNG this way, it will
likely run slower than one unique PRNG.

It has been suggested to use short-period MTs with different
characteristic polynomials. This is also available for the nVidia GPU
using CUDA. This is probably the generator I would pick if I wanted to
produce random numbers in parallel. However, here we should be aware
that the math has not been thoroughly proven.

http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/DC/dgene.pdf
http://developer.download.nvidia.com/compute/cuda/2_2/sdk/website/projects/MersenneTwister/doc/MersenneTwister.pdf

There is also a version of MT that use SIMD instructions internally,
which gives a factor-of-two speedup.

I would never use different algorithms. It will introduce a systematic
difference in the simulation.



-- 
http://mail.python.org/mailman/listinfo/python-list

ANNOUNCE: awstats_reader 0.5

[Joshua Kugler]

ABOUT THE MODULE

AwstatsReader is a pythonic interface to AWStats data cache files.  Using
it, you can access year and month via dict-like subscripts, and and
individual data points via both dict-like subscripts and attribute-like
accessors.

As of version 0.5, it includes a script for merging AWStats Cache files.

Download: http://azariah.com/open_source.html

ABOUT THE AUTHOR

Joshua Kugler (jos...@azariah.com) is a programmer and system administator
with over 10 years of industory experience.  He is currently looking for a
job.  Happen to have one you could offer him? :)
Resume at: http://jjncj.com/papers/KuglerResume.pdf

DISCLAIMER
==
This is an early-beta release.  There are 43 tests which cover most, if not
all of the functionality, but not much documentation.  The interface should
be considered stable, but not in concrete.  The usage of this project in
a "real world" situation (awstats_cache_merge.py) led to many improvements
to the API.

I wrote this via examples from an AWStats cache file, so I'm sure there are
sections for which I do not have definitions.  If you would send me those
sections, I'll be sure to add them.

Right now, this will parse and display cache files from AWStats 6.5. I've
not tested other versions yet, as 6.5 is the only version I've had access
to so far.

REPOSITORY
==
No public repository yet. Just haven't set it up.

LICENSE
===
Modified BSD

EXAMPLE
===
import awstats_reader

obj  = awstats_reader.AwstatsReader('/path/to/awstats_logs', 'example.com')

print obj[2007]
print obj[2008][6]
m = obj[2009][7]
print m['general']
# Access like a dictionary...
print m['general']['LastLine']
#...or like an object attribute
print m['general'].LastLine
print m.general.LastLine

FEEDBACK

Please send questions/comments/suggestions to awstatsrea...@azariah.com
For now, you can find the latest version here:
http://azariah.com/open_source.html


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[sturlamolden]

On 19 Des, 22:58, sturlamolden  wrote:

> If you pick two random states (using any PRNG), you need error-
> checking that states are always unique, i.e. that each PRNG never
> reaches the starting state of the other(s).

Another note on this:

Ideally, we would e.g. know how to find (analytically) MT states that
are very far apart. But to my knowledge no such equation has been
derived.

But often in Monte Carlo simulations, the PRNG is not the dominant
computational bottleneck. So we can simply start N PRNGs from N
consequtive states, and for each PRNG only use every N-th pseudorandom
deviate.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sort the values of a dict

[Rory Campbell-Lange]

> mylist = [z for z in zip(list(d), list(d.values()))]
> The Pythonic way for the above line is:
> >>> mylist = list(d.items())

Quite right. Thanks for pointing that out. 

I liked Chris Kaynor's solution:

s = sorted(d.items(), key=lambda i: i[1][2])

I'm a bit rusty, and have just picked up the new "Learning Python" book
but I'm finding that it isn't providing a useful refresher to the
language or a particularly good introduction to version 3. Have you any
suggestions.

Regards
Rory

-- 
Rory Campbell-Lange
r...@campbell-lange.net

Campbell-Lange Workshop
www.campbell-lange.net
0207 6311 555
3 Tottenham Street London W1T 2AF
Registered in England No. 04551928
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Carl Johan Rehn]

On 19 Dec, 23:09, sturlamolden  wrote:
> On 19 Des, 22:58, sturlamolden  wrote:
>
> > If you pick two random states (using any PRNG), you need error-
> > checking that states are always unique, i.e. that each PRNG never
> > reaches the starting state of the other(s).
>
> Another note on this:
>
> Ideally, we would e.g. know how to find (analytically) MT states that
> are very far apart. But to my knowledge no such equation has been
> derived.
>
> But often in Monte Carlo simulations, the PRNG is not the dominant
> computational bottleneck. So we can simply start N PRNGs from N
> consequtive states, and for each PRNG only use every N-th pseudorandom
> deviate.

Thank you for pointing me to the short-period MT reference and
especially the reference on the CUDA-version of parallel MT (even
though I would have wished the author had included a benchmark
comparison in the report). This is a very interesting topic. I agree
that it may work to start PRNGs at distinct and different states, but
that bookkeeping may slow down the algorithm so that it is not worth
the effort. However, the CUDA-version sounds interesting and should be
easy enough to use in a practical application.

Carl
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sort the values of a dict

[John Bokma]

Rory Campbell-Lange  writes:

> I'm a bit rusty, and have just picked up the new "Learning Python" book
> but I'm finding that it isn't providing a useful refresher to the
> language or a particularly good introduction to version 3. Have you any
> suggestions.

While you aren't addressing me, I do have 2 suggestions:

 * Programming in Python 3 by Mark Summerfield (make sure you get the
   2nd edition)
 * Dive into Python 3 by Mark Pilgrim

I am halfway in the first book (but the 1st edition) and like it a
lot. I know Dive into Python from an earlier version (2.x) and have no
problem at all recommending the Python 3 edition. You can read/download it
here: http://diveintopython3.org/

-- 
John Bokma

Read my blog: http://johnbokma.com/
Hire me (Perl/Python): http://castleamber.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

a python downloader for the Amazon mp3 store

[Chris Colbert]

So, I wasn't happy with the Amazon mp3 downloader for linux (because it
sucks). And clamz is written in C with a bunch of dependencies.

Thus, I've created a python downloader for .amz files using the crypto keys
figured out by Ben Moody (clamz author).

Its just command line only right now, but I will add a pyqt front end in the
future.

The only external dependency is PyCrypto, which is in the ubuntu repos.

I always appreciate feedback!

GPL licensed
http://code.google.com/p/pymazon/

Cheers!

Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[mdipierro]

On Dec 19, 12:42 am, AppRe Godeck  wrote:
> Just curious if anybody prefers web2py over django, and visa versa. I
> know it's been discussed on a flame war level a lot. I am looking for a
> more intellectual reasoning behind using one or the other.

Of course I am the most biased person in the world on this topic but
perhaps you want to hear my bias.

A little bit of history... I thought a Django course at DePaul
University and built a CMS for the United Nations in Django. I loved
it. Then I also learned RoR. I found RoR more intuitive and better for
rapid prototyping. I found Django much faster and more solid. I
decided to build a proof of concept system that was somewhat in
between Django and Rails with focus on 3 features: 1) easy to start
with (no installation, no configuration, web based IDE, web based
testing, debugging, and database interface); 2) enforce good practice
(MVC, postbacks); 3) secure (escape all output, talk to database via
DAL to present injections, server-side cookies with uuid session keys,
role based access control with pluggable login methods, regex
validation for all input including URLs).

Originally it was a proof of concept, mostly suitable for teaching.
Then lots of people helped to make it better and turn it into a
production system. Now he had more than 50 contributors and a more
than 20 companies that provide support.

There are some distinctive features of web2py vs Django. Some love
them, some hate hate them (mostly people who did not try them):

- We promise backward compatibility. I do not accept patches that
break it. It has been backward compatible for three years and I will
enforce the copyright, if necessary, in order to ensure it for the
future.

- In web2py models and controllers are not modules. They are not
imported. They are executed. This means you do not need to import
basic web2py symbols. They are already defined in the environment that
executes the models and controllers (like in Rails). This also means
you do not need to restart the web server when you edit your app. You
can import additional modules and you can define modules if you like.

- You have a web based IDE with editor, some conflict resolution,
Mercurial integration, ticketing system, web-based testing and
debugging.

- The Database Abstraction Layer (DAL) is closed to SQL than Dango ORM
is. This means it does less for you (in particular about many 2 many)
but it is more flaxible when it comes to complex joins, aggregates and
nested selects.

- The DAL supports out of the box SQLite, MySQL, PostgreSQL, MSSQL,
Oracle, FireBird, FireBase, DB2, Informix, Ingres, and the Google App
Engine (except for joins and multi-entity transactions). We plan
support for Sybase and MongoDB within one month.

- The DAL supports transactions. It means it will create and/or ALTER
tables for you as your model changes. This can be disabled.

- The DAL has partial support for some legacy databases that do not
have an 'id' auto increment primary key.

- It has a plugin and a component systems (here is an old video:
http://www.vimeo.com/7182692 the video says "experimental" but this is
now stable in trunk, although not very well documented).

- both systems have a web based database interface (Django calls it
"admin", web2py calls it "appadmin, not to be confused with web2py
"admin", the IDE). The Django one is more polished, customizable and
designed to be exposed to users. The web2py one is raw and designed
for the administrator. It is not customizable. Because it is designed
for the administrator and requires administrator login it allows
arbitrary DAL code to be executed. It can be disabled.

Here is the last app I built with it: http://www.vimeo.com/7182692
running on GAE here: http://www.vimeo.com/7182692


Here http://www.vimeo.com/6507384 you can see a video in which I
rewrite the Django polls tutorial in web2py. You will get an idea of
some of the differences.

Anyway, I think both system are great. Spend 15 minutes (no more) with
each to make up your mind, and stick with it.

Massimo
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[mdipierro]

Errata. I said "The dal supports transactions" where I meant "the dal
supports migrations".
Of course it also supports "transactions" as well as "distributed
transactions".


On Dec 19, 5:32 pm, mdipierro  wrote:
> On Dec 19, 12:42 am, AppRe Godeck  wrote:
>
> > Just curious if anybody prefers web2py over django, and visa versa. I
> > know it's been discussed on a flame war level a lot. I am looking for a
> > more intellectual reasoning behind using one or the other.
>
> Of course I am the most biased person in the world on this topic but
> perhaps you want to hear my bias.
>
> A little bit of history... I thought a Django course at DePaul
> University and built a CMS for the United Nations in Django. I loved
> it. Then I also learned RoR. I found RoR more intuitive and better for
> rapid prototyping. I found Django much faster and more solid. I
> decided to build a proof of concept system that was somewhat in
> between Django and Rails with focus on 3 features: 1) easy to start
> with (no installation, no configuration, web based IDE, web based
> testing, debugging, and database interface); 2) enforce good practice
> (MVC, postbacks); 3) secure (escape all output, talk to database via
> DAL to present injections, server-side cookies with uuid session keys,
> role based access control with pluggable login methods, regex
> validation for all input including URLs).
>
> Originally it was a proof of concept, mostly suitable for teaching.
> Then lots of people helped to make it better and turn it into a
> production system. Now he had more than 50 contributors and a more
> than 20 companies that provide support.
>
> There are some distinctive features of web2py vs Django. Some love
> them, some hate hate them (mostly people who did not try them):
>
> - We promise backward compatibility. I do not accept patches that
> break it. It has been backward compatible for three years and I will
> enforce the copyright, if necessary, in order to ensure it for the
> future.
>
> - In web2py models and controllers are not modules. They are not
> imported. They are executed. This means you do not need to import
> basic web2py symbols. They are already defined in the environment that
> executes the models and controllers (like in Rails). This also means
> you do not need to restart the web server when you edit your app. You
> can import additional modules and you can define modules if you like.
>
> - You have a web based IDE with editor, some conflict resolution,
> Mercurial integration, ticketing system, web-based testing and
> debugging.
>
> - The Database Abstraction Layer (DAL) is closed to SQL than Dango ORM
> is. This means it does less for you (in particular about many 2 many)
> but it is more flaxible when it comes to complex joins, aggregates and
> nested selects.
>
> - The DAL supports out of the box SQLite, MySQL, PostgreSQL, MSSQL,
> Oracle, FireBird, FireBase, DB2, Informix, Ingres, and the Google App
> Engine (except for joins and multi-entity transactions). We plan
> support for Sybase and MongoDB within one month.
>
> - The DAL supports transactions. It means it will create and/or ALTER
> tables for you as your model changes. This can be disabled.
>
> - The DAL has partial support for some legacy databases that do not
> have an 'id' auto increment primary key.
>
> - It has a plugin and a component systems (here is an old 
> video:http://www.vimeo.com/7182692the video says "experimental" but this is
> now stable in trunk, although not very well documented).
>
> - both systems have a web based database interface (Django calls it
> "admin", web2py calls it "appadmin, not to be confused with web2py
> "admin", the IDE). The Django one is more polished, customizable and
> designed to be exposed to users. The web2py one is raw and designed
> for the administrator. It is not customizable. Because it is designed
> for the administrator and requires administrator login it allows
> arbitrary DAL code to be executed. It can be disabled.
>
> Here is the last app I built with it:http://www.vimeo.com/7182692
> running on GAE here:http://www.vimeo.com/7182692
>
> Herehttp://www.vimeo.com/6507384you can see a video in which I
> rewrite the Django polls tutorial in web2py. You will get an idea of
> some of the differences.
>
> Anyway, I think both system are great. Spend 15 minutes (no more) with
> each to make up your mind, and stick with it.
>
> Massimo
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[Jake]

On Dec 19, 1:42 am, AppRe Godeck  wrote:
> Just curious if anybody prefers web2py over django, and visa versa. I
> know it's been discussed on a flame war level a lot. I am looking for a
> more intellectual reasoning behind using one or the other.

Hi!  I come from a mvc framework background in a few different
languages (java, php, ruby).  I'm only a couple weeks into web2py, I'm
finding web2py a joy to work with.  I won't waste your time with the
features it provides, as you can find these on the website.  There is
also a copy of the lead developer's book on the site, and its very
well written.

If web2py intrigues you, I would recommend just trying it for a few
days.  It's pretty small (relatively few exposed classes and
functions) and very comprehensible, so you won't really have to invest
much if you decide to try it and don't like it.  The mailing list is
active and responsive, and the lead developer happens to have chimed
in on every question i've asked.  On the down side, the irc community
is very small.

Good Luck,
Jake



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Gregory Ewing]

Lie Ryan wrote:

If you don't care about "repeatability" (which is already extremely 
difficult in parallel processing even without random number generators), 
you can just start two PRNG at two distinct states (and probably from 
two different algorithms)


There's no need for different algorithms, and no problem
with repeatability. There exist algorithms with a very
long period (on the order of 2*100 or more) for which it's
easy to calculate seeds for a set of guaranteed non-
overlapping subsequences.

http://or.journal.informs.org/cgi/content/abstract/44/5/816
http://or.journal.informs.org/cgi/content/abstract/47/1/159

In these kinds of generator, moving from one state to
the next involves multiplying a state vector by a matrix
using modulo arithmetic. So you can jump ahead N steps
by raising the matrix to the power of N.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw string substitution problem

[Rhodri James]

On Fri, 18 Dec 2009 17:58:08 -, Alan G Isaac   
wrote:



On 12/17/2009 7:59 PM, Rhodri James wrote:

"re.compile('a\\nc')" passes a sequence of four characters to
re.compile: 'a', '\', 'n' and 'c'.  re.compile() then does it's own
interpretation: 'a' passes through as is, '\' flags an escape which
combined with 'n' produces the newline character (0x0a), and 'c' passes
through as is.



I got that from MRAB's posts. (Thanks.)
What I'm not getting is why the replacement string
gets this particular interpretation.  What is the payoff?


So that the substitution escapes \1, \2 and so on work.

--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[mdipierro]

Errata 2. Before people jump on me. I said "copyright" but I meant
"trademark". Of course web2py is GPL2 so everybody can copy and modify
it.

The license has an exception that basically treats the compiled web2py
as freeware.

The license does not extend to apps that require web2py. They can be
distributed under any license you like, included closed source and
bundled with the web2py binary.

On Dec 19, 5:32 pm, mdipierro  wrote:
> On Dec 19, 12:42 am, AppRe Godeck  wrote:
>
> > Just curious if anybody prefers web2py over django, and visa versa. I
> > know it's been discussed on a flame war level a lot. I am looking for a
> > more intellectual reasoning behind using one or the other.
>
> Of course I am the most biased person in the world on this topic but
> perhaps you want to hear my bias.
>
> A little bit of history... I thought a Django course at DePaul
> University and built a CMS for the United Nations in Django. I loved
> it. Then I also learned RoR. I found RoR more intuitive and better for
> rapid prototyping. I found Django much faster and more solid. I
> decided to build a proof of concept system that was somewhat in
> between Django and Rails with focus on 3 features: 1) easy to start
> with (no installation, no configuration, web based IDE, web based
> testing, debugging, and database interface); 2) enforce good practice
> (MVC, postbacks); 3) secure (escape all output, talk to database via
> DAL to present injections, server-side cookies with uuid session keys,
> role based access control with pluggable login methods, regex
> validation for all input including URLs).
>
> Originally it was a proof of concept, mostly suitable for teaching.
> Then lots of people helped to make it better and turn it into a
> production system. Now he had more than 50 contributors and a more
> than 20 companies that provide support.
>
> There are some distinctive features of web2py vs Django. Some love
> them, some hate hate them (mostly people who did not try them):
>
> - We promise backward compatibility. I do not accept patches that
> break it. It has been backward compatible for three years and I will
> enforce the copyright, if necessary, in order to ensure it for the
> future.
>
> - In web2py models and controllers are not modules. They are not
> imported. They are executed. This means you do not need to import
> basic web2py symbols. They are already defined in the environment that
> executes the models and controllers (like in Rails). This also means
> you do not need to restart the web server when you edit your app. You
> can import additional modules and you can define modules if you like.
>
> - You have a web based IDE with editor, some conflict resolution,
> Mercurial integration, ticketing system, web-based testing and
> debugging.
>
> - The Database Abstraction Layer (DAL) is closed to SQL than Dango ORM
> is. This means it does less for you (in particular about many 2 many)
> but it is more flaxible when it comes to complex joins, aggregates and
> nested selects.
>
> - The DAL supports out of the box SQLite, MySQL, PostgreSQL, MSSQL,
> Oracle, FireBird, FireBase, DB2, Informix, Ingres, and the Google App
> Engine (except for joins and multi-entity transactions). We plan
> support for Sybase and MongoDB within one month.
>
> - The DAL supports transactions. It means it will create and/or ALTER
> tables for you as your model changes. This can be disabled.
>
> - The DAL has partial support for some legacy databases that do not
> have an 'id' auto increment primary key.
>
> - It has a plugin and a component systems (here is an old 
> video:http://www.vimeo.com/7182692the video says "experimental" but this is
> now stable in trunk, although not very well documented).
>
> - both systems have a web based database interface (Django calls it
> "admin", web2py calls it "appadmin, not to be confused with web2py
> "admin", the IDE). The Django one is more polished, customizable and
> designed to be exposed to users. The web2py one is raw and designed
> for the administrator. It is not customizable. Because it is designed
> for the administrator and requires administrator login it allows
> arbitrary DAL code to be executed. It can be disabled.
>
> Here is the last app I built with it:http://www.vimeo.com/7182692
> running on GAE here:http://www.vimeo.com/7182692
>
> Herehttp://www.vimeo.com/6507384you can see a video in which I
> rewrite the Django polls tutorial in web2py. You will get an idea of
> some of the differences.
>
> Anyway, I think both system are great. Spend 15 minutes (no more) with
> each to make up your mind, and stick with it.
>
> Massimo

-- 
http://mail.python.org/mailman/listinfo/python-list

Class variables static by default?

[KarlRixon]

Given the following script, I'd expect p1.items to just contain
["foo"] and p2.items to contain ["bar"] but they both contain ["foo",
"bar"].

Why is this? Are object variables not specific to their instance?

---
#!/usr/bin/env python

class Parser:
items = []
def add_item(self, item):
self.items.append(item)

p1 = Parser()
p1.add_item("foo")
p2 = Parser()
p2.add_item("bar")

print p1
print p2
print p1.items
print p2.items


Output:
<__main__.Parser instance at 0x7fd812ccc098>
<__main__.Parser instance at 0x7fd812ccc0e0>
['foo', 'bar']
['foo', 'bar']
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how do I set a Python installation as the default under windows ?

[Steve Holden]

Stef Mientki wrote:
> hello,
> 
> I just upgraded from Python 2.5 to 2.6.
> Most of the things work,
> but I'm struggling with one issue,
> when I start Python in a command window,
> it still uses Python 2.5.
> 
> Is there a way to get Python 2.6 as my default Python environment ?
> 
> thanks,
> Stef Mientki

It's a matter of replacing C:\Python25 with C:\Python26 in your PATH
environment variable, which is what the Windows command processor uses
to fined executable programs. It's normal to add both the directory the
interpreter lives in /and/  its Scripts subdirectory, so you may have
two replacements to make. See

  http://www.python.org/doc/faq/windows/

for more. If anyone has any good updates to that document I'll be happy
to make them (or give you permission to make them ...). It was written a
long time ago, and I'm not sure it's had much love. In particular
there'll not be anything relating to Windows 7.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Steven D'Aprano]

On Sat, 19 Dec 2009 09:02:38 -0800, Carl Johan Rehn wrote:

> Well, in Matlab I used "tic; for i = 1:1000, randn(100, 1), end;
> toc" and in IPython i used a similar construct but with "time" instead
> of tic/(toc.


I don't know if this will make any significant difference, but for the 
record that is not the optimal way to time a function in Python due to 
the overhead of creating the loop sequence.

In Python, the typical for-loop construct is something like this:

for i in range(1, 1000)

but range isn't a funny way of writing "loop over these values", it 
actually creates a list of integers, which has a measurable cost. In 
Python 2.x you can reduce that cost by using xrange instead of range, but 
it's even better to pre-compute the list outside of the timing code. Even 
better still is to use a loop sequence like [None]*1000 (precomputed 
outside of the timing code naturally!) to minimize the cost of memory 
accesses.

The best way to perform timings of small code snippets in Python is using 
the timeit module, which does all these things for you. Something like 
this:

from timeit import Timer
t = Timer('randn(100, 1)', 'from numpy import randn')
print min(t.repeat())

This will return the time taken by one million calls to randn. Because of 
the nature of modern operating systems, any one individual timing can be 
seriously impacted by other processes, so it does three independent 
timings and returns the lowest. And it will automatically pick the best 
timer to use according to your operating system (either clock, which is 
best on Windows, or time, which is better on Posix systems).



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Question about dir function

[Ray Holt]

When I run a dir(_builtins_) I get the error message that the name
_builtins_ is not defined. I haven't tried the dir function on other
functions, but can someone tell me why I am getting this message? Thanks,
Ray
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class variables static by default?

[John Posner]

On Sat, 19 Dec 2009 19:10:13 -0500, KarlRixon  wrote:


Given the following script, I'd expect p1.items to just contain
["foo"] and p2.items to contain ["bar"] but they both contain ["foo",
"bar"].

Why is this? Are object variables not specific to their instance?

---
#!/usr/bin/env python

class Parser:
items = []
def add_item(self, item):
self.items.append(item)





You're using a *class attribute* instead of an *instance attribute*.
Change the class definition to:

class Parser:
def __init__(self):
self.items = []

def add_item(self, item):
self.items.append(item)

-John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Class variables static by default?

[Xavier Ho]

Yes, if you want instance variables that are unique to each instance of a
class, do the following:

class Parser:
def __init__(self):
   self.items = []

And that should work fine.

J:\_Programming Projects\Python>python test.py
<__main__.Parser object at 0x0240E7B0>
<__main__.Parser object at 0x02411930>
['foo']
['bar']

Cheers,
Xav

On Sun, Dec 20, 2009 at 10:10 AM, KarlRixon  wrote:

> Given the following script, I'd expect p1.items to just contain
> ["foo"] and p2.items to contain ["bar"] but they both contain ["foo",
> "bar"].
>
> Why is this? Are object variables not specific to their instance?
>
> ---
> #!/usr/bin/env python
>
> class Parser:
>items = []
>def add_item(self, item):
>self.items.append(item)
>
> p1 = Parser()
> p1.add_item("foo")
> p2 = Parser()
> p2.add_item("bar")
>
> print p1
> print p2
> print p1.items
> print p2.items
> 
>
> Output:
> <__main__.Parser instance at 0x7fd812ccc098>
> <__main__.Parser instance at 0x7fd812ccc0e0>
> ['foo', 'bar']
> ['foo', 'bar']
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about dir function

[Matt Nordhoff]

Ray Holt wrote:
> When I run a dir(_builtins_) I get the error message that the name
> _builtins_ is not defined. I haven't tried the dir function on other
> functions, but can someone tell me why I am getting this message?
> Thanks, Ray

You are getting that message because the name "_builtins_" is not
defined, as it says. You were probably looking for "__builtins__", with
2 underscores on each end, not just 1.

BTW, "__builtins__" is a CPython implementation detail. If you want to
play with built-in objects, you should import the __builtin__ (no "s")
module and use that.
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class variables static by default?

[Cameron Simpson]

On 19Dec2009 16:10, KarlRixon  wrote:
| Given the following script, I'd expect p1.items to just contain
| ["foo"] and p2.items to contain ["bar"] but they both contain ["foo",
| "bar"].
| 
| Why is this? Are object variables not specific to their instance?

You haven't instatiated "items" in the object instance, so python finds
it further out in the class. One class, one variable; it looks "static".

Compare this (yours):

  class Parser:
items = []
def add_item(self, item):
  self.items.append(item)

with this:

  class Parser:
def __init__(self):
  self.items = []
def add_item(self, item):
  self.items.append(item)

The first makes an "items" in the class namespace.

The second makes an "items" in each object as it is initialised.

Run the rest of your code as is and compare.

When you say "self.items" python looks first in the object and then in
the class. Your code didn't put it in the object namespace, so it found
the one in the class.

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

I have always been a welly man myself. They are superb in wet grass, let
alone lagoons of pig shit.  - Julian Macassey
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class variables static by default?

[Lie Ryan]

On 12/20/2009 11:10 AM, KarlRixon wrote:

Given the following script, I'd expect p1.items to just contain
["foo"] and p2.items to contain ["bar"] but they both contain ["foo",
"bar"].

Why is this? Are object variables not specific to their instance?


First of all, dump all the preconception of what 'static' and 'class 
variable' means in any previous language you've learned. Those terms is 
not the same in python, and not even referring to a similar concept.


In python, 'class variable' is a variable that belongs to a class; not 
to the instance and is shared by all instance that belong to the class.


In contrast, 'instance variable' belongs to the instance, and each 
instance can make their instance variables refers to different objects 
than the other instances.


Note that, in python, an object can be "owned" by multiple variables, or 
more precisely "an object can have multiple names".


'static' in python refers to 'staticmethod', these are called 
'classmethod' in other languages (C/C++/Java). Python's 'classmethod' is 
an entirely different beast than 'classmethod' in other languages.


'staticmethod' in python is a method in a class that is not bound to the 
class nor the instance; the class is merely used as organizational tool. 
'classmethod' in python is a method that is bound to the class instead 
of the instance; the first argument (self/cls) of a 'classmethod' is 
bound to the class instead of the instance.

--
http://mail.python.org/mailman/listinfo/python-list

Please Help Publicize PyCon

[Steve Holden, Chairman, PSF]

Hi,everyone.

This year I hope all readers of this list will assist me in crass
commercial promotion of next year's PyCon. I will be speaking about
"Building the Python Community", and we can't do that without
advertising that the Python community exists (and hey, has pretty good
conferences too).

One particularly effective way for you prodigious email producers to
assist is to something to your signature (as you will see I have done).
This is especially important because it will bring PyCon to the
attention of a wider audience - "preaching to the choir" is all very
well, but doesn't build the community.

regards
 Steve
-- 
Steve HoldenChairman, Python Software Foundation
The Python Community Conference   http://python.org/psf/
PyCon 2010 Atlanta Feb 19-21http://us.pycon.org/
Watch PyCon on video now!  http://pycon.blip.tv/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about dir function

[Tim Chase]

Ray Holt wrote:

When I run a dir(_builtins_) I get the error message that the name
_builtins_ is not defined. I haven't tried the dir function on other
functions, but can someone tell me why I am getting this message? Thanks,
Ray



So close, and yet thrown by requisite extra underscores:

>>> dir(_builtins_) # one _ before and after
Traceback (most recent call last):
  File "", line 1, in 
NameError: name '_builtins_' is not defined
>>> dir(__builtins__) # two _'s before and after
['ArithmeticError', 'AssertionError', 'AttributeError', 
'BaseException', 'DeprecationWarning', 'EOFError', 'Ellipsis', 
'EnvironmentError', 'Exception', 'False',...]


-tkc


--
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Lie Ryan]

On 12/20/2009 8:58 AM, sturlamolden wrote:

On 19 Des, 21:26, Lie Ryan  wrote:


you can just start two PRNG at two distinct states


No. You have to know for certain that the outputs don't overlap.


Not necessarily, you only need to be certain that the two streams don't 
overlap in any reasonable amount of time. For that purpose, you can use 
a PRNG that have extremely high period like Mersenne Twister and puts 
the generators to very distant states.



If you pick two random states (using any PRNG), you need error-
checking that states are always unique, i.e. that each PRNG never
reaches the starting state of the other(s). If I had to do this, I
would e.g. hash the state to a 32 bit integer. You can deal with
errors by jumping to a different state or simply aborting the
simulation. The problem is that the book-keeping will be costly
compared to the work involved in generating a single pseudorandom
deviate. So while we can construct a parallel PRNG this way, it will
likely run slower than one unique PRNG.


You will need some logic to ensure that your generator's states are 
distant enough, but that won't incur any generation overhead. Of course 
this relies on the very large period of the PRNG algorithm.

--
http://mail.python.org/mailman/listinfo/python-list

Checker 1.0 Released!

[Chris Withers]

I'm pleased to announce the first release of Checker.

This is a cross-platform, pluggable tool for comparing the configuration 
of a machine with a known configuration stored in text files in a source 
control system all written in Python.


For more information, please see:
http://www.simplistix.co.uk/software/python/checker

cheers,

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk




--
http://mail.python.org/mailman/listinfo/python-list

Re: numpy performance and random numbers

[Steven D'Aprano]

On Sat, 19 Dec 2009 13:58:37 -0800, sturlamolden wrote:

> On 19 Des, 21:26, Lie Ryan  wrote:
> 
>> you can just start two PRNG at two distinct states
> 
> No. You have to know for certain that the outputs don't overlap.

"For certain"? Why?

Presumably you never do a Monte Carlo simulation once, you always repeat 
the simulation. Does it matter if (say) one trial in fifty is lower-
quality than the rest because the outputs happened to overlap? What if 
it's one trial in fifty thousand? So long as the probability of overlap 
is sufficiently small, you might not even care about doing repeated 
simulations.


> If you pick two random states (using any PRNG), you need error- checking
> that states are always unique, i.e. that each PRNG never reaches the
> starting state of the other(s). If I had to do this, I would e.g. hash
> the state to a 32 bit integer. You can deal with errors by jumping to a
> different state or simply aborting the simulation. The problem is that
> the book-keeping will be costly compared to the work involved in
> generating a single pseudorandom deviate. So while we can construct a
> parallel PRNG this way, it will likely run slower than one unique PRNG.

Since truly random sequences sometimes produce repeating sequences, you 
should be able to tolerate short periods of overlap. I'd consider the 
following strategy: for each of your parallel PRNGs other than the first, 
periodically jump to another state unconditionally. The periods should be 
relatively prime to each other, e.g. the second PRNG jumps-ahead after 
(say) 51 calls, the third after 37 calls, etc. (the exact periods 
shouldn't be important). Use a second, cheap, PRNG to specify how far to 
jump ahead. The overhead should be quite small: a simple counter and test 
for each PRNG, plus a small number of calls to a cheap PRNG, and 
statistically you should expect no significant overlap.

Is this workable?



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How Do I...?

[Steve Holden, Chairman, PSF]

Victor Subervi wrote:
> On Sat, Dec 19, 2009 at 10:22 AM, Tim Chase
> mailto:python.l...@tim.thechases.com>>
> wrote:
> 
> Victor Subervi wrote:
> 
> On Fri, Dec 18, 2009 at 3:03 PM, Tim Chase
>  >wrote:
> 
> Well, you start by reading a book on how to program.  You
> would then learn
> that what you want (in all likelihood) is a dictionary/map
> structure for
> dynamically created key/value pairs. Once you have
> progressed from your
> current apprenticeship and achieved the rank of third-degree
> journeyman
> programmer, the ways of dynamic variable creation will avail
> themselves.
> 
> 
> Why the arrogance? Why talk down to me?
> 
> 
> The aim was not arrogance, but expression of exasperation 
> 
> 
> Exasperation is what triggers your arrogance. Learn to control it. "Walk
> a mile in my mocassins." You can't do it. I'm an artist. I think out of
> my right hemisphere, not my left like you. You couldn't possibly
> understand. Yes, sometimes I chide myself for the questions I ask when I
> see the answers come back. The problem is that I quite literally can't
> think like you. I have to force myself to do it every time. To you it's
> as natural as breathing, which is why you can't relate.
> 
It's perhaps not the most helpful approach to characterize as arrogance
what is the actually the result of ignorance that only you could cure
(since it requires some knowledge of your personal circumstances). It's
not unusual to assume that everyone on this list has programming as a
strong component of their skill set.

I haven't been around the list much lately, so I don't know if your
earlier posts were ever preceded by "please help a struggling
right-brain artist", but on this group it certainly wouldn't have hurt.

> Thank you for your help anyway. Thank you for your patience. Please try
> to understand. It starts by understanding you can't understand. You have
> my continued promise that I will do all I can to edit my questions as
> intelligently as you would before I post them. Trust me, I don't like
> looking foolish, and I know I do. You should recognize that that alone
> is chiding enough. It doesn't stop me, however, for continuing to
> program. The whole universe is a program. I program to understand it in
> a way you couldn't even begin to grasp.
> 
Well, if we were looking for arrogance we could easily interpret that
last statement as such. Please remember that although we are mainly
left-brain types on this list some of us do have artistic and musical
skills. Some people will therefore probably be closer to your conception
than you anticipate (though I confess I am unlikely to be one of them).

> BTW, although I know full well about dictionaries, I didn't know one
> could use them like you indicated, nor would I have known how to google
> it. I did try googling, FYI.
> V
> 
The PSF has recently started to take diversity more seriously, and this
should ideally include cultural diversity in all sorts of dimensions, so
I hope you will stick around long enough to inject the artistic approach
from time to time. Python has a *very* diverse user list, but not all
are equally represented in the on-line communities.

Tim is correct, the Python list is a pretty friendly place as far as
netiquette standards go, and I am happy this thread didn't just turn
into ugly name calling (which it could have with a less adult approach
by the participants).

regards
 Steve
-- 
Steve HoldenChairman, Python Software Foundation
The Python Community Conference   http://python.org/psf/
PyCon 2010 Atlanta Feb 19-21http://us.pycon.org/
Watch PyCon on video now!  http://pycon.blip.tv/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sort the values of a dict

[Rhodri James]

On Fri, 18 Dec 2009 23:00:42 -, David Robinow   
wrote:



I won't engage in any arguments about pythonicity but it seems simpler
if you convert to a list of tuples right away.

d = {1:('a', 1, 12), 5:('r',21,10), 2:('u',9,8)}
l = [(x, d[x]) for x in d.keys()]


which is a long way of writing "l = d.items()", or "l = list(d.items())"  
if you're using Python 3. :-)



def third(q):
return q[1][2]

s = sorted(l, key=third)
print s



--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

doing cool stuff with Python and Industrial Robots....

[Chris Colbert]

Im just finishing up some research work during a stint as a visiting
researcher in Germany.

I've made a short clip showing a KUKA robot performing object reconstruction
using a single camera mounted on the robot.

The entire system is written in Python (control, math, everything) and
related infrastructure (Cython/NumPy/Scipy/Mayavi/etc...)

I cant give any technical details yet, as I have a few papers still pending
publication. But I thought you all might enjoy the video.

Everyone likes big robots!!!

~60MB
www.therealstevencolbert.com/dump/reco_demo.mpg

The video begins by showing the ground-truth of the object rendered as a
wireframe. The robot then captures three images of the object from various
vantages. The software then reconstructs the shape of the object, and
overlays the wireframe with the results.


Cheers!

Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

using time.gov to get the current time

[Rick]

Is there any way to get the current time from time.gov using urllib2 or 
something similar?

Does anyone have any examples of doing this?

Thanks!
Rick King
Southfield MI

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dangerous behavior of list(generator)

[Albert van der Horst]

In article ,
Gabriel Genellina  wrote:

>
>Despite a promise in PEP 289, generator expressions semantics isn't
>explained in detail in the language reference. I can't tell if the
>difference is intentional, accidental, undocumented behavior, an
>implementation accident, a bug, or what...

Philosophically speaking ...
An important feature that is not documented is a severe defect.
(important maps to severe).
Before it is documented, there can be no discrepancy between
specification and implementation so other defects are formally
not present in relation to this situation.

>--
>Gabriel Genellina
>

Groetjes Albert.


--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyArg_ParseTupleAndKeywords in Python3.1

[Joachim Dahl]

My mistake seems to be that I declared

char a, b;

instead of

int a, b;

Thank you for sorting this out.

Joachim
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: iterators and views of lists

[Carl Banks]

On Dec 18, 12:18 pm, "Alf P. Steinbach"  wrote:
[lots of tangential information snipped]
> ...but I don't understand why you're limiting yourself to the STL.

Because I believe the vast majority of iterators used in C++ in
practice are the ones provided by STL types, therefore I felt a
statement about the STL iterators reflected the commonly used power
(if not the ultimate capability) of iterators in C++.  I suppose it
was a bit hasty.


> ... "ostream_iterator" ...

Well that's a good example.

I retract my statement that C++ iterators in general not having this
power, but I still doubt that C++ iterators are used like this much.
In most code I've seen people use STL or similar data structures, and
then use the iterators for the particular data type, and/or pointers.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Using ZODB (or something else) for storing lots of metadata about RSS/Atom feeds and posts

[Thomas Doggette]

I'm working on (or rather, at this point, planning) a project that
will involve keeping track of every post in a large number of Atom
feeds, as well as a lot of metadata about how posts are linked and how
users interact with them.

The idea of having all of these be persistent objects is very
appealing, but I'm not sure the ZODB is ideal for what I'm doing. Can
people with more experience working with data sets like this give me
some advice on what they've had luck using?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using time.gov to get the current time

[Matt Nordhoff]

Rick wrote:
> Is there any way to get the current time from time.gov using urllib2 or 
> something similar?
> 
> Does anyone have any examples of doing this?
> 
> Thanks!
> Rick King
> Southfield MI

Why isn't your local system's clock accurate enough?
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I look for proxy cache like apt-proxy (for Debian Package) but for python eggs package…

[Steve Holden]

Diez B. Roggisch wrote:
> Klein Stéphane schrieb:
>> Hi,
>>
>> I look for a tools to do proxy cache like apt-proxy (for Debian
>> Package) but for python eggs package.
>>
>> Can a easy-install option perform this feature ?
> 
> No. But you might install EggBasket, which is a PyPI-like server.
> 
>   http://www.chrisarndt.de/projects/eggbasket/
> 
> However, it is *not* a proxy, and to the best of my knowledge that's not
> easily done anyway due to the way easy_install is working. It scrapes
> the website it is pointed to, and when it finds something it likes,
> follows that.
> 
> Thus the server doesn't get an idea *what* easy_install is looking for,
> and thus can't relay the request to the "real" PyPI, fetching the egg,
> storing it locally, and then re-deliver it.
> 
> Instead what we do with eggbasket is to copy all the eggs we obtained by
> other means into a central repository that he serves. Thus we have them
> backed-up & available even if PyPI or the actual package go away.
> 
> Diez

Time that software came out of beta! What needs fixing before a release?

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: iterators and views of lists

[Rhodri James]

On Fri, 18 Dec 2009 21:10:28 -, Brendan Miller   
wrote:



When I said they are "weak" I meant it in sense that the algorithms
writeable against an InputerIterator interface (which is what python's
iterator protocol provides) is a proper subset of the algorithms that
can be written against a RandomAccessIterator interface. The class of
algorithms expressible against a python iterator is indeed limited to
those that can be expressed with a for each loop or map/reduce
operation.


I think the most relevant thing that can be said to this is that you need  
to stop trying to write Python code as if it was C++.  The languages  
constructs are intended for different purposes; iterators in C++ are part  
of the small arsenal of devices needed to provide type-independence to  
algorithms, something that comes a lot more easily to Python through  
dynamic typing and easy slicing.  I'm barely literate in C++, but I can't  
see uses for a C++ random access iterator that aren't served in Python by  
simple indexing (possibly with additional bounds checking) or a Python  
iterator (possibly over a sliced sequence).


The downside, of course, is that you don't have the security provided by  
static type checking.  You pays your money, you takes your choice;  
preferably, however, you don't do like a friend of mine and try to write  
FORTRAN in whatever language you're using!


--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: using time.gov to get the current time

[Matt Nordhoff]

Matt Nordhoff wrote:
> Rick wrote:
>> Is there any way to get the current time from time.gov using urllib2 or 
>> something similar?
>>
>> Does anyone have any examples of doing this?
>>
>> Thanks!
>> Rick King
>> Southfield MI
> 
> Why isn't your local system's clock accurate enough?

Bah, I'm better at IRC than e-mail... Anyway:

The easiest way to get highly-accurate time is with an "apt-get install
ntp" (or your OS's equivalent), and that solves it for all of your
applications, not just this one.

The time synchronization solutions that exist today are the result of
decades of development by experts. If you try to write some solution
yourself, it will not be as accurate (though that's okay if it still
meets your accuracy requirements), and you're liable to make some
mistake* and wind up with a bunch of angry NTP server operators chasing
after you with torches and pitchforks.

* Such as making excessively-frequent requests, or making more frequent
requests if the server appears down, or ignoring NTP Kiss-o'-Death
packets. Do not do these things!
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[AppRe Godeck]

On Sat, 19 Dec 2009 12:48:07 -0800, Yarko wrote:

> On Dec 19, 12:42 am, AppRe Godeck  wrote:
>> Just curious if anybody prefers web2py over django, and visa versa. I
>> know it's been discussed on a flame war level a lot. I am looking for a
>> more intellectual reasoning behind using one or the other.
> 
> Chevy or Ford?  (or whatever pair you prefer) vi or emacs?
> ...
> 
> These hold one aspect.
> 
> Hammer or a saw?
> 
> Hold (perhaps) another...
> 

tis the nature of man kind to war.

> us.pycon.org, for example, uses both (in reality a mix of the above
> argument sets, but at least evidence of the latter: different tools for
> different problems).
> 
> From a rapid prototyping perspective, web2py is heavily data-table
> efficient: that is, you can define a system, and all the app creation,
> form generation and validation have defaults out of the box, and you can
> have a "sense" of your data-centric structure in minutes.   The same
> argument can go against ("how do I get it to do exactly what _I_ want it
> to, not what it wants to?") - that is, defaults hide things, and  that
> has two edges...
> 
> From a layout/user interaction rapid prototyping perspective, web2py is
> just entering the waters...
> 

> There is a steady growth of users, and (as you would expect for a young
> framework), a lot of changes going on (although backward compatiblity is
> a constant mantra when considering changes, that too is a double-edged
> thing).
> 
> I find web2py useful, fast, and at times / in areas not as evolved /
> flexible as I'd like.  BUT I could learn it quickly, and get to work
> quickly.
> 
> I have taken an intro Django course (at a PyCon), have built a few
> things with it (not nearly as many as I have w/ web2py), and I _can_ do
> things in it - so I'll let someone else w/ django "miles" under their
> belt speak their mind.
> 
> - Yarko

It seems that this is the biggest issue surrounding web2py, from my 
research, is the ability to customize the defaults (the easy). If all 
web2py offers is default views, then it may be good for proof of concept 
projects, however I can't see in my right mind, proofing an application, 
and then turning around to write it in django because more than the 
defaults is needed.

Thanks for your replies, I was hoping to hear from some django people as 
well. Especially if you choose django over web2py, and why.

If you have time to write, 

Give a situation where you would prefer to have django over web2py, and 
then another situation you would prefer web2py over django, and why.

Why does web2py have classes that represent HTML? I can't see ever 
needing to write VIEW code in my controller, since thats what views are 
for.

It seems that even though web2py is fast, supports lots of features, the 
fact that in the end it gets in the way of doing what you want is it's 
downfall. Django, even though requiring more "ground work", this ground 
work becomes a solid foundation on which to build your application on.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating Classes

[Dave Angel]

Steve Holden wrote:

Dave Angel wrote:
  

seafoid wrote:


Hey Guys,

I have started to read over classes as a brief respite from my parsing
problem.

When a class is defined, how does the class access the data upon which
the
class should act?

Example:

class Seq:   
def __init__(self, data, alphabet = Alphabet.generic_alphabet):

self.data = data self.alphabet = alphabet

def
tostring(self):  
return self.data 
def tomutable(self):

return MutableSeq(self.data, self.alphabet)
def count(self, item):
return len([x for x in self.data if x == item])

I know what it should do, but have no idea how to feed it the data.

Methinks I need to invest in actual computing books as learning from
biologists is hazy!

Kind regards,
Seafoid.

  

Steve's message was good, but I feel he kind of jumped in the middle.  A
class is a description of a type of object, and the behaviors and data
that each instance of the object supports.

You create the object by using the class name like a function call.  The
arguments to that "call" are passed to the __init__ method.

So  obj = Seq("abcd")or   obj = Seq("defg", "abcdefg")would each
create an object of the class.  But generally, many objects will exist,
each with different data.

The data in the object is accessed in what appears to be the "self"
namespace.  The name self is just a convention, but it's the first
argument of each method of the class.  So when somebody calls the
count() method, they pass 'item' a value, but self is used to refer to
that particular object's data.

So how does 'self' get assigned?  That's what Steve was describing. 
When you use the syntax:

obj.count("value")

you actually call the count method with self referring to "obj" and item
referring to "value".  obj does double-duty here, both defining which
class' count() method will be called, and also supplying the first
parameter to the call, the "self" parameter.

There are more complex things that can go on, like creating "bound"
function objects, but  I think this should get you pretty far.

One other point:  you should always derive a class from some other
class, or 'object' by default.  So you should being the class definition
by:

class Seq(object):

Why?  It mainly has to do with super().  But in any case if you omit the
'object' it's an "old style" class, and that's not even supported in
3.x, so it's better to just get in the habit before it matters.



With respect, unless you have to expound on the differences between the
old-style and the new-style classes (which aren't relevant here) you are
just introducing a red herring by even mentioning it. The average Python
user won't need to use super() in their first year as a Python programmer.

And, since you brought up Python 3, it's not necessary to explicitly
inherit from object to get new-style classes because, as you correctly
point out, old-style classes don't exist in Python 3.

I have no idea why you think "you should always derive a class from some
other class". That's pretty unnecessary.

regards
 Steve
  
I'm not sure why, but since it changes behavior (more than just 
super()), and since the old behavior is deprecated, I think it's 
worthwhile to use new-style classes.  And although you don't need to 
explicitly  do it in Python 3.x, it does no harm.


DaveA


--
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[mdipierro]

> Why does web2py have classes that represent HTML? I can't see ever
> needing to write VIEW code in my controller, since thats what views are
> for.

You do not need to use but if, for example, you want to build a menu
recursively, having a server-side presentation of the DOM allows to do
it without string manipulation. It is safer and less error prone.
Anyway, it is not something you must use.

Lots of the features are optional. Like the web based IDE. If you do
not like it, you can use the shell like you use Django.

 > It seems that even though web2py is fast, supports lots of
features, the
> fact that in the end it gets in the way of doing what you want is it's
> downfall. Django, even though requiring more "ground work", this ground
> work becomes a solid foundation on which to build your application on.

What do you mean by "gets in the way"? Could you provide an example?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Moving from PHP to Python. Is it Possible

[mdipierro]

About you point 3). You may want to look into:

http://www.web2py.com/php

It translates a PHP page into a web2py template. It is crude and
primitive and fails in some cases. Moreover a literal translation is
not what you really want since you want to follow a more MVC design.
Moreover it will not always maps PHP functions into valid Python/
web2py ones. Yet it may be a useful as a learning tool. I made it to
convert Drupal templates into web2py layouts.

Massimo


On Dec 11, 4:26 am, Sancar Saran  wrote:
> Greetings.
>
> I'm 35 yrs old self learner  and who do daily PHP coding for food more than a
> decade.
>
> After ten years of PHP coding I'm getting bored and give try for learning new
> things.
>
> After 3 days of crawling google, diving in python and cursing, now  I can show
> something on my linux/apache/mod_wsgi setup.
>
> And i'm struck on something.
>
> I had CMS design. It works with PHP very well. And I want to transfer my
> design in Python.
>
> My design depends on a Global Array. A huge array which store everything about
> requested Web page for a final rendering.
>
> In PHP accessing globals is easy. You may do direct access or use class
> something like ZEND Registry.
>
> I'm looking for similar facility in python world.
>
> Also I have couple of questions.
>
> 1-) Can I create Global (read/write access anywhere from my code) Multi
> dimensional, associative array (or hash) and store any kind of variable type.
>
> 2-) Is there any error trigger for mod_wsgi. When something go bad I god
> Internal Server error. Looking for error log was't nice. Is there any way to
> shows errors in the web page ?
>
> 3-) Any documents books sites about helping moving from php to python
>
> 4-) In php we had someting like
> ob_start(); // start Output buffering
> require('hede.php'); // include phtml similar to psp. mixed language and html
> (to avoiding string replacement (or marker based) html templates)
> $var = ob_get_clean(); // get rendered output and use anywhere
>
> can find a similar abilities in Python ?
>
> 5-) is there any Python documentation based on examples. When I give up about
> to finding  php's $_REQUEST or $_SERVER equivalents in python some demo code 
> in
> Twisted docs are much helpful than any other words. Me and my kind already
> have  problem with English language. Example code much more helpful than full
> academic description.
>
> Thanks for support and patience for this old noob.
>
> Regards...

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class variables static by default?

[Steven D'Aprano]

On Sun, 20 Dec 2009 11:44:11 +1100, Lie Ryan wrote:

> In python, 'class variable' is a variable that belongs to a class; not
> to the instance and is shared by all instance that belong to the class.

Surely, since string variables are strings, and float variables are 
floats, and bool variables are bools, and module variables are modules, a 
class variable will be a class and an instance variable will be an 
instance?

> In contrast, 'instance variable' belongs to the instance, and each
> instance can make their instance variables refers to different objects
> than the other instances.

The usual term used in Python is "class attribute" and "instance 
attribute" for the named fields of a class or instance.




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[Thadeus Burgess]

On Dec 19, 8:39 pm, AppRe Godeck  wrote:
> On Sat, 19 Dec 2009 12:48:07 -0800, Yarko wrote:
> > On Dec 19, 12:42 am, AppRe Godeck  wrote:
> >> Just curious if anybody prefers web2py over django, and visa versa. I
> >> know it's been discussed on a flame war level a lot. I am looking for a
> >> more intellectual reasoning behind using one or the other.
>
> > Chevy or Ford?  (or whatever pair you prefer) vi or emacs?
> > ...
>
> > These hold one aspect.
>
> > Hammer or a saw?
>
> > Hold (perhaps) another...
>
> tis the nature of man kind to war.
>
>
>
>
>
> > us.pycon.org, for example, uses both (in reality a mix of the above
> > argument sets, but at least evidence of the latter: different tools for
> > different problems).
>
> > From a rapid prototyping perspective, web2py is heavily data-table
> > efficient: that is, you can define a system, and all the app creation,
> > form generation and validation have defaults out of the box, and you can
> > have a "sense" of your data-centric structure in minutes.   The same
> > argument can go against ("how do I get it to do exactly what _I_ want it
> > to, not what it wants to?") - that is, defaults hide things, and  that
> > has two edges...
>
> > From a layout/user interaction rapid prototyping perspective, web2py is
> > just entering the waters...
>
> > There is a steady growth of users, and (as you would expect for a young
> > framework), a lot of changes going on (although backward compatiblity is
> > a constant mantra when considering changes, that too is a double-edged
> > thing).
>
> > I find web2py useful, fast, and at times / in areas not as evolved /
> > flexible as I'd like.  BUT I could learn it quickly, and get to work
> > quickly.
>
> > I have taken an intro Django course (at a PyCon), have built a few
> > things with it (not nearly as many as I have w/ web2py), and I _can_ do
> > things in it - so I'll let someone else w/ django "miles" under their
> > belt speak their mind.
>
> > - Yarko
>
> It seems that this is the biggest issue surrounding web2py, from my
> research, is the ability to customize the defaults (the easy). If all
> web2py offers is default views, then it may be good for proof of concept
> projects, however I can't see in my right mind, proofing an application,
> and then turning around to write it in django because more than the
> defaults is needed.
>
> Thanks for your replies, I was hoping to hear from some django people as
> well. Especially if you choose django over web2py, and why.
>
> If you have time to write,
>
> Give a situation where you would prefer to have django over web2py, and
> then another situation you would prefer web2py over django, and why.
>
> Why does web2py have classes that represent HTML? I can't see ever
> needing to write VIEW code in my controller, since thats what views are
> for.
>
> It seems that even though web2py is fast, supports lots of features, the
> fact that in the end it gets in the way of doing what you want is it's
> downfall. Django, even though requiring more "ground work", this ground
> work becomes a solid foundation on which to build your application on.

You need to realize that web2py is still a relatively young framework,
issues to certain problems are currently being addressed in web2py.
For example, the DAL is being re-written in a much more modular maner,
allowing the addition of new database backends to be quite seemless.
Other parts of web2py will follow as web2py grows I am sure. I
wouldn't disqualify web2py based off what you read, you really need to
try it for yourself, and if you run into a problem, I'm sure, some way
there is a solution if you bring up the problem, and your problem
might even help web2py to grow to be an even better framework.

I dare you to try developing a web application with both. Spend one
day working on a simple django application, polls, blog, image
gallery, family pet tree, you name it. Then take the next day, and
write the same application with web2py, and you decide. In the end,
both are tools and you need to figure out what is best for YOU.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class variables static by default?

[Chris Rebert]

On Sat, Dec 19, 2009 at 8:16 PM, Steven D'Aprano
 wrote:
> On Sun, 20 Dec 2009 11:44:11 +1100, Lie Ryan wrote:
>
>> In python, 'class variable' is a variable that belongs to a class; not
>> to the instance and is shared by all instance that belong to the class.
>
> Surely, since string variables are strings, and float variables are
> floats, and bool variables are bools, and module variables are modules, a
> class variable will be a class and an instance variable will be an
> instance?

As they say, the exception proves the rule. :)

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anybody use web2py?

[Anand Vaidya]

On Dec 19, 2:42 pm, AppRe Godeck  wrote:
> Just curious if anybody prefers web2py over django, and visa versa. I
> know it's been discussed on a flame war level a lot. I am looking for a
> more intellectual reasoning behind using one or the other.

Hi,

I am not very familiar with Django, anyway, my reasons for selecting
web2py are:

- I believe Django naturally "fits in" to a publishing type of
application. web2py seems to be more focussed on being a front-end to
"applications" not so much for CMS type or newspaper type publishing.
(There is a web2py based wiki/CMS app, though). Though, I agree either
could fulfil any of these roles.

- Django documentation is vastly superior, including third party books
etc. After the v2 Web2py book, we do have solid documentation, but
web2py evolves so quickly, there are always things that are documented
only on the google groups, slices or wiki.

- Many training courses are available pretty much everywhere for
Django. Web2py needs to catch up - it will, probably.

- Web2py lowers the barrier to entry into python web programming,
since it is well thought out and intuitive. The support on the mailing
list is fantastic, though I have no idea how good the support for
Django is. w2py is easy to pick up for even a python newbie.

- Massimo claims about the backward compatibility. I was surprised
recently when I took an app I deployed several months (and several
versions old) and retested it with the latest SVN code. And it worked
fine! Which means, if my client asks for it, I could swap out old w2py
with the latest code with no problems.

My $0.02
-- 
http://mail.python.org/mailman/listinfo/python-list

tkinter import problem

[harish anand]

Hi,
I have Mandriva 2010.0 in my laptop.
I installed python3.1 from the repository.
But i am unable to import tkinter in python console.
When I try to import tkinter I get the following error,
`ImportError : No module named _tkinter`

Am I doing something wrong?

Thanks,
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: iterators and views of lists

[Anh Hai Trinh]

On Dec 20, 12:04 am, Anh Hai Trinh  wrote:

> chain:
>
>   sorted(itertools.chain(listagent(x)[::2], listagent(y)[-1:1:-2]))
>   [0, 4, 8, 12, 13, 15, 16, 17, 19]
>
> zip:
>
>   sorted(itertools.izip(listagent(z)[1::3], listagent(x)[2::3]))
>   [(452, 16), (758, 4), (898, 10)]

I think I mis-interpret Andrei's slides. I think what he meant to sort
a chain of slices is such that to move the smaller elements into the
first-given slices in-place, thus moving items around whatever lists
underlying those slices.

And for zip, I think he meant sort the first slice, but moving in-
place the items referred by the others in lock-step.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programming intro book ch1 and ch2 (Windows/Python 3) - Request For Comments

[Alf P. Steinbach]

* John Posner:
On Fri, 18 Dec 2009 13:00:48 -0500, Alf P. Steinbach  
wrote:




Chapter 2 is about Basic Concepts (of programming). It's the usual: 
variables, ...


1. Overall suggestion

You have a tendency to include non-pertinent asides [1]. But then, 
rambling a bit endows a manuscript with the author's distinctive voice. 
Fortunately, we live in a hypertext-enabled world, where you can have 
your cake and eat it, too. I suggest that you go over your manuscript 
with a ruthless eye, and turn your rambles into hypertext-accessible 
"sidebars". See how much you can reduce the length of Chapter 2, which 
current runs 98 pages!


He he. :-)

You're right, although the examples you chose were not as I see it rambles, just 
short asides within sentences; useful concrete examples; and general background.


I do have a tendency to apparently ramble, especially about technical details 
and consequences  --  it's not arbitrary but an attempt to bring in important 
consequences and/or some more fundamental unifying view. I tried to discipline 
myself to not do that, to just let go, to let /the reader/ replace arbitrary 
rules with understanding, in time. Like, you don't really need to understand the 
physics to do hammering. Even though even a rough understanding of the physics 
can help you avoid looking like an effeminate Hollywood movie star acting out 
hammering, can help you avoid getting hurt, can get the job done much faster, 
and can help you in other situations! But I'll try even harder. :-)




2. Comments on Section 2.6.7, "References & automatic garbage collection"

There's a spell-check evader on page 2:49: "trough" s.b. "through". And 
your spell-checker should have caught "adressable" on page 2:48.


Sad to say, I'm not using a spell checker. The reason is that for me, in /most/ 
cases the speling erors marked by Word's spell checker are not speling erors. 
The beast is Just Wrong. And it's Wrong-Headed, insisting on "fixing" perfectly 
correct text. For example, replacing "southeast" with "southwest" (now, what the 
 does Word's spell checker know about geography? Nothing!) Even worse, as 
soon as that beast just gets its little toe inside the door, it tends to invite 
in its friends the Grammar Checker and the Auto Formatter. Which feel free to 
not only report what they see as improvements, but to just change the text!


So, thanks!

Will be fixed.


I find your sequence-of-attribute-lookups approach to the topic of 
"variable assignment" interesting. The directed-graph illustration on 
page 2:49 even hints at the fact that in the world of Python, names 
("turtle", "forward", etc.) and objects (various kinds of yellow boxes) 
are very different things.


(I suggest getting rid of the callout "Very small fixed size variable". 
That way, only objects, not names, have the italicized callouts.)


But using the term "references" and the directed-graph metaphor has its 
drawbacks. Avoiding the term "reference" will make it easier to navigate 
the turbulent waters of call-by-reference vs. call-by-value vs. 
call-by-name. (You don't even stick a toe in those waters in Section 
2.7.5.) Combining memory addresses with the directed graph metaphor 
invites the reader to think at way too low a level, IMHO.


Uhm, discussing call-by-name would really be rambling & anecdotal. AFAIK no real 
programming language has had that since Algol, where it was introduced by a too 
unclear spec implemented literally (i.e., lack of 2-way communication within the 
team introduced a "feature"!). Or am I misunderstanding what you mean?


It's True, as you note, that people tend to get confused. I recently (just weeks 
ago) was astounded to see that a C++ "expert" thought that Java had pass by 
reference, apparently because in Java only references are passed around. But I 
think the cure for that is not to draw a veil over the things that can confuse, 
such as avoiding the word "reference", but rather to just discuss the reality.


That is, I think argument passing & associated terminology is not more 
complicated than even my mom could understand, if she put her politician's mind 
to it. Certainly, anyone young enough won't have any problem. I believe... ;-)



[snip]


I hope these comments help.


Yes, thanks,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list